6 Commits

Author SHA1 Message Date
sebastien lussier
284f8b47db World Partition - Builder Commandlet: A collection name can now be provided as an argument instead of a map name.
* Builder will execute on each map of the collection
#rb jeanfrancois.dube
#preflight 641127d45819afacafdae915

[CL 24648808 by sebastien lussier in ue5-main branch]
2023-03-14 23:22:53 -04:00
Tim Smith
b3eae65080 Updating sample build scripts to not build UnrealHeaderTool
#rb
#preflight 63c01af8bf54fa7b3668d1d7

[CL 23662742 by Tim Smith in ue5-main branch]
2023-01-12 09:57:35 -05:00
sebastien lussier
c0753f3011 Updated the HLOD generation BuildGraph example script to include most changes from our internal demo
#rb jeanfrancois.dube
#preflight trivial
#jira none

#ROBOMERGE-AUTHOR: sebastien.lussier
#ROBOMERGE-SOURCE: CL 18475313 in //UE5/Release-5.0/... via CL 18481104
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18481114 by sebastien lussier in ue5-release-engine-test branch]
2021-12-16 19:01:27 -05:00
sebastien lussier
8086487a85 World Partition HLOD builder - Run the setup phase with -AllowCommandletRendering
#rb richard.malo

#ROBOMERGE-SOURCE: CL 16840322 via CL 16841786
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16841889 by sebastien lussier in ue5-release-engine-test branch]
2021-07-13 16:08:43 -04:00
sebastien lussier
d272103bf6 In Distributed HLOD builds, allow -SetupHLODs & -BuildHLODs steps to run without source control
* Will prevent AWS buildmachines from holding an exclusive lock on actor files in case of a crash & if the Horde process doesn't cleanup checked out file properly (often the case)
* All p4 operations were temporary during those two steps those anyway
* Much faster setup step

The last step, -SubmitHLODs will obviously require source control to be enabled
#rb jeanfrancois.dube

#ROBOMERGE-OWNER: sebastien.lussier
#ROBOMERGE-AUTHOR: sebastien.lussier
#ROBOMERGE-SOURCE: CL 16365501 in //UE5/Private-Frosty/... via CL 16370860
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v804-16311228)

[CL 16370880 by sebastien lussier in ue5-release-engine-test branch]
2021-05-18 13:28:28 -04:00
sebastien lussier
945039e0b1 WorldPartitionHLODsBuilder
* Added options to generate/use a build manifest
* -BuildManifest=Filename.ini
* When calling with -SetupHLODs, provide "-BuildManifest=Filename.ini -BuilderCount=N" to have N workloads in the build manifest
* When calling with -BuildHLODs, provide "-BuildManifest=Filename.ini -BuilderIdx=i" to build workload i

Added UWorldPartitionBuilder::PreWorldInitialization()

Added support for distributed World Partition HLOD generation
* Runs multiple steps on different machines
* Setup: Done on a single machine, will create/update/delete HLOD actors as required. The distributed setup step takes a "BuilderCount" argument, which will be used to write a build manifest that specify the work that needs to be done by the N builders
* Build: Performed on multiple machines, read work to be done from the build manifest
* Submit: Final step, perform on a single machine. Gather files modified in all the previous steps and submit them in a single changelist

Intermediary files that are created/modified/deleted are transfered between the jobs as build products, and submitted in the last step.

To run with UAT:
RunUAT.bat BuildGraph -Script=Engine\Build\Graph\Examples\BuildWorldPartitionHLODs.xml -Target="HLOD Generation" -set:ProjectName=YourProjectName -set:EditorTarget=EditorTargetName -set:MapName=YourMapName -set:BuilderCount=N

To run steps individually:
* Setup (N = Builder count to use)
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -SetupHLODs -DistributedBuild -BuilderCount=N
* Build (i from 0 to N-1):
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -DistributedBuild -BuildHLODs -BuilderIdx=i
* Submit:
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -DistributedBuild -SubmitHLODs

Changes integrated from UE5 main:
#ushell-cherrypick of 15669199 by Sebastien.Lussier
#ushell-cherrypick of 15687351 by JeanFrancois.Dube
#ushell-cherrypick of 15693052 by Sebastien.Lussier
#ushell-cherrypick of 15693777 by Sebastien.Lussier

#ROBOMERGE-SOURCE: CL 15710613 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v781-15675533)

[CL 15710619 by sebastien lussier in ue5-main branch]
2021-03-16 08:56:01 -04:00