Commit Graph

228 Commits

Author SHA1 Message Date
Ryan Durand
74a82706a9 Catch exceptions when scanning folders for folders that need to be deleted.
#rnx
#rb none
#jira none

[CL 15696383 by Ryan Durand in ue5-main branch]
2021-03-15 10:25:55 -04:00
Sebastien Lussier
102ce82546 Added support for distributed World Partition HLOD generation
* Runs multiple steps on different machines
* Setup: Done on a single machine, will create/update/delete HLOD actors as required. The distributed setup step takes a "BuilderCount" argument, which will be used to write a build manifest that specify the work that needs to be done by the N builders
* Build: Performed on multiple machines, read work to be done from the build manifest
* Submit: Final step, perform on a single machine. Gather files modified in all the previous steps and submit them in a single changelist

Intermediary files that are created/modified/deleted are transfered between the jobs as build products, and submitted in the last step.

To run with UAT:
RunUAT.bat BuildGraph -Script=Collaboration\Frosty\Frosty\Build\FrostyHLOD.xml -Target="HLOD Generation" -set:MapName=YourMapName -set:HLODBuilderCount=N

To run steps individually:
* Setup (N = Builder count to use)
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -SetupHLODs -DistributedBuild -BuilderCount=N
* Build (i from 0 to N-1):
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -DistributedBuild -BuildHLODs -BuilderIdx=i
* Submit:
    YourProject YourMapName -run=WorldPartitionBuilderCommandlet -SCCProvider=Perforce -AllowCommandletRendering -Builder=WorldPartitionHLODsBuilder -DistributedBuild -SubmitHLODs


#fyi jeanfrancois.dube, richard.malo, vincent.beauchemin

[CL 15693777 by Sebastien Lussier in ue5-main branch]
2021-03-14 21:58:48 -04:00
ben marsh
ff498fd482 Fix incorrect hostname for PCB submit workspace.
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 15685133 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v781-15675533)

[CL 15685153 by ben marsh in ue5-main branch]
2021-03-11 23:17:52 -04:00
ben marsh
d53e84576b Include a hash of the server address when creating a workspace for submitting binaries. Should fix issues with different edge servers being selected.
#rb none
#jira UE-109866

#ROBOMERGE-SOURCE: CL 15681548 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v781-15675533)

[CL 15684115 by ben marsh in ue5-main branch]
2021-03-11 21:31:36 -04:00
ben marsh
b4553ed744 Fix permission bits being reset by heurisitc that retrieves files from temp storage.
#rb none
#jira UE-109089

#ROBOMERGE-SOURCE: CL 15676777 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v781-15675533)

[CL 15676799 by ben marsh in ue5-main branch]
2021-03-11 11:19:28 -04:00
Ryan Durand
b6eff43263 Fixing compile break.
#rnx
#rb none
#jira none

[CL 15675703 by Ryan Durand in ue5-main branch]
2021-03-11 08:21:47 -04:00
Ryan Durand
517d36846d Catch and log exceptions when enumerating files/folders to delete.
#rnx
#rb none
#jira none

[CL 15675515 by Ryan Durand in ue5-main branch]
2021-03-11 07:27:29 -04:00
Ryan Durand
71b89da3ae Report an error if we fail to delete an empty folder but continue working.
#rnx
#rb none
#jira none

[CL 15672472 by Ryan Durand in ue5-main branch]
2021-03-10 21:40:02 -04:00
ben marsh
e2fd50118e Remove legacy Mono unzip path.
#rb none
#jira UE-109036

#ROBOMERGE-SOURCE: CL 15669463 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v779-15635321)

[CL 15670868 by ben marsh in ue5-main branch]
2021-03-10 18:23:35 -04:00
Ryan Durand
168200dff5 Add support for shunting Unzip tasks to the legacy unzip logic.
#rnx
#rb none
#jira none

[CL 15649177 by Ryan Durand in ue5-main branch]
2021-03-08 20:02:46 -04:00
ben marsh
f010f289e4 Use the cross platform zip methods to store files in temp storage.
#jira UE-109089
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 15646046 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v779-15635321)

[CL 15646062 by ben marsh in ue5-main branch]
2021-03-08 17:32:34 -04:00
ben marsh
a4d4c5c295 Fix for missing file permissions on Mac.
Switches the default zip utility to be the built in NET ZipArchive class. Adds additional support for storing permission bits by P/Invoking libSystem.dylib and adding the flags from stat64() to the ZipArchiveEntry. Also restores the permissions on unzip. One less dependency on Ionic.Zip. :)

#jira UE-109089
[FYI] Jonathan.Adamczewski
#rb none

#ROBOMERGE-SOURCE: CL 15620859 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v777-15581079)

[CL 15620862 by ben marsh in ue5-main branch]
2021-03-04 22:18:07 -04:00
Marc Audy
9753392e2b Merge UE5/RES CL# 15462083 to UE5/Main
This represents UE4/Main @ 15414221

[CL 15463811 by Marc Audy in ue5-main branch]
2021-02-18 18:13:28 -04:00
benoit deschenes
08076c32d7 Integrating CL# 15321861 from //UE4/Main
#jira UE-108402
#rb none

[CL 15388736 by benoit deschenes in ue5-main branch]
2021-02-11 12:34:27 -04:00
Marc Audy
cac1fe0019 Merge UE5/Release-Engine-Staging @ CL# 15299266 to UE5/Main
This represents UE4/Main @ CL# 15277572

[CL 15299962 by Marc Audy in ue5-main branch]
2021-02-03 14:57:28 -04:00
Ryan Durand
764dded8db Fix compile errors in WaitTask.cs.
#rnx
#rb none
#jira none

[CL 15187253 by Ryan Durand in ue5-main branch]
2021-01-25 17:13:41 -04:00
Ryan Durand
852d3639dc Adding WaitTask.
#rnx
#rb none
#jira none

[CL 15183849 by Ryan Durand in ue5-main branch]
2021-01-25 15:53:37 -04:00
Marc Audy
bf80889353 UE5/Release-Engine-Staging to UE5/Main
This represents UE4/Main up to CL# 14958402

[CL 15028197 by Marc Audy in ue5-main branch]
2021-01-08 19:56:07 -04:00
Ben Marsh
068f97616c Restore packages before building projects. Should fix IncludeTool errors in CIS.
#rb none
#rnx

[CL 14964192 by Ben Marsh in ue5-main branch]
2020-12-23 11:23:34 -04:00
Ben Marsh
6f927647b1 Changing C# code over to using EpicGames.Core over DotNETCommon.
#rb none
#rnx

[CL 14962096 by Ben Marsh in ue5-main branch]
2020-12-21 23:07:37 -04:00
Ben Marsh
64651ad903 Remove references to FastJSON functionality in DotNETCommon. Use a duplicate copy in AutomationScripts called Json_DEPRECATED instead.
#rb none
#rnx

[CL 14962028 by Ben Marsh in ue5-main branch]
2020-12-21 22:05:50 -04:00
Ben Marsh
69646e6e96 Convert fastJSON into a standalone thirdparty dependency. Allows converting the DotNETUtilities and BuildUtilities folders to NET Core 3.1.
#rb none
#rnx

[CL 14961920 by Ben Marsh in ue5-main branch]
2020-12-21 19:26:55 -04:00
Ben Marsh
572812e3f2 BuildGraph: Downgrade invalid argument errors to a warning.
#rb none

[CL 14906564 by Ben Marsh in ue5-main branch]
2020-12-11 16:14:17 -04:00
Marc Audy
ada7c144fa Merge //UE5/Release-Engine-Staging @14903491 to //UE5/Main
[CL 14906022 by Marc Audy in ue5-main branch]
2020-12-11 14:21:20 -04:00
Joakim Lindqvist
017048f505 Changed pdb format to not use portable pdbs (which is the default in net core) and instead back to our old pdb configuration.
This resolves a issue with PCBs were pdbcopy failed to run because it does not support portable pdbs.

#rb none

[CL 14837521 by Joakim Lindqvist in ue5-main branch]
2020-12-02 14:54:53 -04:00