Commit Graph

45 Commits

Author SHA1 Message Date
sebastien lussier
fae787c8d3 World Partition HLOD builder - Run the setup phase with -AllowCommandletRendering
#rb richard.malo


#ROBOMERGE-SOURCE: CL 16840322
#ROBOMERGE-BOT: (v836-16769935)

[CL 16841786 by sebastien lussier in ue5-main branch]
2021-07-13 16:05:19 -04:00
ryan durand
9a8b0a5c40 Use an empty "complete" tag set to signal cook completion for package nodes so that test agents don't implicitly require all the cooked assets.
#rnx
#rb none


#ROBOMERGE-SOURCE: CL 16703343
#ROBOMERGE-BOT: (v835-16672529)

[CL 16703350 by ryan durand in ue5-main branch]
2021-06-17 08:12:45 -04:00
logan harvell
e742a81d21 Added options for uploading a binary using the online BuildPatchTool
#ROBOMERGE-SOURCE: CL 16663746
#ROBOMERGE-BOT: (v834-16658389)

[CL 16663835 by logan harvell in ue5-main branch]
2021-06-14 16:32:35 -04:00
Jerome Delattre
0471a03acd Fix 'Update Version Files' defined several times when building from more than one platform
#jira UE-117576
#rnx
#rb trivial

[CL 16588593 by Jerome Delattre in ue5-main branch]
2021-06-08 12:31:58 -04:00
Jerome Delattre
9af966d299 Fix local project output path when using BuildAndTestProject buildgraph
#jira UE-117280
#rnx
#rb Andrew.Grant

[CL 16574018 by Jerome Delattre in ue5-main branch]
2021-06-07 13:42:07 -04:00
yuriy odonnell
632398d0ae Use buffered file IO during unsync manifest generation step
#ROBOMERGE-SOURCE: CL 16505152 in //UE5/Private-Frosty/...
#ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v826-16501804)

[CL 16505159 by yuriy odonnell in ue5-main branch]
2021-05-29 10:18:55 -04:00
sebastien lussier
aecfb0d95d In Distributed HLOD builds, allow -SetupHLODs & -BuildHLODs steps to run without source control
* Will prevent AWS buildmachines from holding an exclusive lock on actor files in case of a crash & if the Horde process doesn't cleanup checked out file properly (often the case)
* All p4 operations were temporary during those two steps those anyway
* Much faster setup step

The last step, -SubmitHLODs will obviously require source control to be enabled
#rb jeanfrancois.dube

#ROBOMERGE-OWNER: sebastien.lussier
#ROBOMERGE-AUTHOR: sebastien.lussier
#ROBOMERGE-SOURCE: CL 16365501 in //UE5/Private-Frosty/...
#ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228)
#ROBOMERGE-CONFLICT from-shelf

[CL 16370860 by sebastien lussier in ue5-main branch]
2021-05-18 13:27:35 -04:00
logan harvell
ea3473cf40 Added options to specify editor/target platforms to skip tests and separated them between boot tests and regular tests
#fyi Jeff.Farris

[CL 16322800 by logan harvell in ue5-main branch]
2021-05-13 20:20:15 -04:00
logan harvell
d7f314ae0b Added build graph option to update version files
#fyi Jeff.Farris

[CL 16145534 by logan harvell in ue5-main branch]
2021-04-28 12:44:15 -04:00
jonathan adamczewski
d35f484540 BuildEditorAndTools.xml:
Recursively bundle pre-compiled AutomationTool binaries, based on the method used in InstalledEngineBuild.xml

(AutomationTool is pre-built before the script is run - there is no need to build it again)


#jira UE-114327

[CL 16119044 by jonathan adamczewski in ue5-main branch]
2021-04-26 17:28:20 -04:00
David Hibbitts
065687ff98 Remove PS5 from "PlatformsToSkipPackaging" to allow PS5 packaging on Frosty
#fyi andrew.grant, jeff.farris

[CL 16052129 by David Hibbitts in ue5-main branch]
2021-04-19 12:47:16 -04:00
Ryan Durand
b285a3e03b Add support for overriding agent types.
#rnx
#rb none
#jira none

[CL 15977138 by Ryan Durand in ue5-main branch]
2021-04-12 09:36:42 -04:00
Andrew Grant
c816a0bf60 Modified BuildAndTest project script to allow projects to save hashed staging data to the network.
Added options for Linux to BuildAndTest and UE5 Horde templates for ShooterGame/QAGame for completeness.

#jira nojira
#rb swarm
#review-15847237 @jeff.farris, @yuriy.odonnell

[CL 15848178 by Andrew Grant in ue5-main branch]
2021-03-28 22:51:06 -04:00
Andrew Grant
c5bf909801 Added option to BuildAndTestProject script to publish staged content
Cleaned up some options around paths to make it easier for scripts to customize them.
Fixed issue with GDKBuilds in Gauntlet where the build source was ignoring the specified recursion depth when searching

#jira nojira
#rb jeff.farris

[CL 15780860 by Andrew Grant in ue5-main branch]
2021-03-23 13:53:51 -04:00
sebastien lussier
945039e0b1 WorldPartitionHLODsBuilder
* Added options to generate/use a build manifest
* -BuildManifest=Filename.ini
* When calling with -SetupHLODs, provide "-BuildManifest=Filename.ini -BuilderCount=N" to have N workloads in the build manifest
* When calling with -BuildHLODs, provide "-BuildManifest=Filename.ini -BuilderIdx=i" to build workload i

Added UWorldPartitionBuilder::PreWorldInitialization()

Added support for distributed World Partition HLOD generation
* Runs multiple steps on different machines
* Setup: Done on a single machine, will create/update/delete HLOD actors as required. The distributed setup step takes a "BuilderCount" argument, which will be used to write a build manifest that specify the work that needs to be done by the N builders
* Build: Performed on multiple machines, read work to be done from the build manifest
* Submit: Final step, perform on a single machine. Gather files modified in all the previous steps and submit them in a single changelist

Intermediary files that are created/modified/deleted are transfered between the jobs as build products, and submitted in the last step.

To run with UAT:
RunUAT.bat BuildGraph -Script=Engine\Build\Graph\Examples\BuildWorldPartitionHLODs.xml -Target="HLOD Generation" -set:ProjectName=YourProjectName -set:EditorTarget=EditorTargetName -set:MapName=YourMapName -set:BuilderCount=N

To run steps individually:
* Setup (N = Builder count to use)
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -SetupHLODs -DistributedBuild -BuilderCount=N
* Build (i from 0 to N-1):
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -DistributedBuild -BuildHLODs -BuilderIdx=i
* Submit:
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -DistributedBuild -SubmitHLODs

Changes integrated from UE5 main:
#ushell-cherrypick of 15669199 by Sebastien.Lussier
#ushell-cherrypick of 15687351 by JeanFrancois.Dube
#ushell-cherrypick of 15693052 by Sebastien.Lussier
#ushell-cherrypick of 15693777 by Sebastien.Lussier

#ROBOMERGE-SOURCE: CL 15710613 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v781-15675533)

[CL 15710619 by sebastien lussier in ue5-main branch]
2021-03-16 08:56:01 -04:00
Andrew Grant
84bfb202fd Fixed case in BuildAndTest graph that could result in no code compilations for a target if a script didn't supply all defaults
#jira #rb na

[CL 15706287 by Andrew Grant in ue5-main branch]
2021-03-15 20:11:03 -04:00
Andrew Grant
a7292aa43c Take a different approach to setting the build location for tests due to some builder weirdness
#jira #rb na

[CL 15690131 by Andrew Grant in ue5-main branch]
2021-03-13 00:08:03 -04:00
Andrew Grant
f3ed6ecb17 Tidied up some options and handle some user-selection-errors better in build script
#jira #rb na

[CL 15688366 by Andrew Grant in ue5-main branch]
2021-03-12 15:39:07 -04:00
Andrew Grant
7602c27306 More BuildGraph tweaks
#jira #rb na

[CL 15685821 by Andrew Grant in ue5-main branch]
2021-03-12 00:44:40 -04:00
Andrew Grant
b2f594bdaa Added some safety checks to build script that will add missing package/build types based on specified tests/packages
#jira #rb na

[CL 15685705 by Andrew Grant in ue5-main branch]
2021-03-12 00:17:33 -04:00
Andrew Grant
9878095f26 Fixed issue locating gauntlet source when building multiple packages into subfolders
#jira #rb na

[CL 15685422 by Andrew Grant in ue5-main branch]
2021-03-11 23:36:19 -04:00
Andrew Grant
98c9555a34 Build scripts -
Added some missing platforms to BuildAndTestProject.xml
Added support for building different configs and MapIniSection
Updated ShooterGame, QAGame, and other build scripts

Exposed options to Horde UI

#jira #rb na

[CL 15685279 by Andrew Grant in ue5-main branch]
2021-03-11 23:26:40 -04:00
Andrew Grant
bf9fa0c8c6 Added Mac options to Topaz tests
Added StarterContent tests

#jira #rb na

[CL 15668918 by Andrew Grant in ue5-main branch]
2021-03-10 15:38:00 -04:00
Andrew Grant
3f1b9834b7 Checking in new scripts for automating tests on projects.
Frosty, Topaz, ShooterGame, and QAGame have helper scripts set up

#jira #rb na

[CL 15662528 by Andrew Grant in ue5-main branch]
2021-03-09 21:05:18 -04:00
Ryan Durand
08540a873a Update BuildEditorAndTools.xml to attempt to get the P4CLIENT from the env vars and use that to generate the submit client, otherwise falling back on the old method.
#rnx
#rb none
#jira UE-109605

[CL 15626974 by Ryan Durand in ue5-main branch]
2021-03-05 13:55:19 -04:00