Commit Graph

62 Commits

Author SHA1 Message Date
joe kirchoff
acf80ab609 [Backout] - CL25694851
[FYI] Joe.Kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: More automated code cleanup

#rnx

[CL 25695155 by joe kirchoff in ue5-main branch]
2023-05-30 19:55:37 -04:00
joe kirchoff
7ed1a39679 UnrealBuildTool: More automated code cleanup
#rnx

[CL 25694864 by joe kirchoff in ue5-main branch]
2023-05-30 19:36:05 -04:00
joe kirchoff
36d266e932 UnrealBuildTool: Automated code cleanup
#rnx
#preflight 6476799e947ff6973c225619

[CL 25693241 by joe kirchoff in ue5-main branch]
2023-05-30 18:38:07 -04:00
joe kirchoff
57a8fe409c UnrealBuildTool: Use expression body for simple properties and accessors
#rnx
#preflight 64766abcfe6a3b258a04cb50

[CL 25691579 by joe kirchoff in ue5-main branch]
2023-05-30 18:01:50 -04:00
tim smith
dce93c0456 1) Added missing exception handling in the tasks
2) Reworked how "stop after error" works
3) Fixed  issue where error propagation might result in a failure to exit.

#rb self
#preflight 645b7ef03172e962d2cf3e47

[CL 25402787 by tim smith in ue5-main branch]
2023-05-10 08:40:51 -04:00
tim smith
e32f741420 1) Addition of the artifact interfaces
2) Moved common code from executors into an immediate action queue.
3) Added support in the immediate action queue to read/write artifacts

#rb joe.kirchoff
#preflight 6454f3c5ed7ffaa0f5b2fbfa

[CL 25355149 by tim smith in ue5-main branch]
2023-05-05 11:19:57 -04:00
joe kirchoff
eeda39b9b9 UnrealBuildTool: Update executor tasks to be long running
#rnx
#rb trivial

[CL 25296539 by joe kirchoff in ue5-main branch]
2023-05-01 20:26:27 -04:00
Ben Marsh
3889217961 UBT: Merging CL 25121561 to support async ToolMode and ActionExecutor instances.
#preflight none

[CL 25121660 by Ben Marsh in ue5-main branch]
2023-04-19 21:44:13 -04:00
joe kirchoff
e0254d98e8 UnrealBuildTool: Fix Action properties that weren't being serialized
#rnx

[CL 24878293 by joe kirchoff in ue5-main branch]
2023-03-31 16:51:20 -04:00
henrik karlsson
3f6b2896a8 [UnrealBuildTool]
* Added experimental executor "Box"

#preflight 6414e9a4ca2afe3ee6c35c9a
#rb partly joe.kirchoff (not the Box code part)

[CL 24702059 by henrik karlsson in ue5-main branch]
2023-03-18 01:42:55 -04:00
bryan sefcik
f3f15a7aa5 Fixed an issue where -allcores would not be used if ProcessorCountMultiplier was set.
Fixed an issue where it was possible to do more parallel actions than cores available.

#rb joe.kirchoff
#preflight 640bc8aa363e9b40abeac263

[CL 24600925 by bryan sefcik in ue5-main branch]
2023-03-10 19:27:40 -05:00
Ben Marsh
7e26c34d2b EpicGames.Core: Deprecate legacy log methods to prefer structured logging interface.
#preflight 640a23d928026468d9de7cbd

[CL 24580590 by Ben Marsh in ue5-main branch]
2023-03-09 14:51:40 -05:00
bryan sefcik
1ad98523c4 Added support to give actions in UBT "weight". "Weight" is an average measurement of how many cores and memory an action uses. Using one core with a normal amount of memory for an action would be a value of 1.
Why do this:
Currently when compiling a cpp file with MSVC, it compiles across multiple cores while clang does not. This means that while we support limiting the number of cores(using ProcessorCountMultiplier), MSVC will use more cores than we specify. It also means that MSVC will always be faster when compiling because clang does not support compiling a cpp over multiple cores. To get similiar results when compiling with clang, we set the weight of MSVC to 1.5 and the weight of clang to 1.0. We then set the ProcessorCountMultiplier to 1.5. This results in MSVC and clang taking roughly the same amount of CPU utilization and clang compiles to be much faster.

	                     Old Timing(secs)	Old CPU Utilization	New Timing	New CPU Utilization(secs)
PlatformA AncientGame	590.94	51	                               431.47	73
MSVC AncientGameEditor	1016.96	94	                             1026.08	95
Clang AncientGameEditor	1543.72	63	                               1270.4	84
PlatformB AncientGame	494	52	                               396.95	74

Old = without weight path
New = with weight path

#jira
#rb christopher.waters, joe.kirchoff
#preflight 6409026c8832f48a4dc72025

[CL 24567859 by bryan sefcik in ue5-main branch]
2023-03-08 17:08:22 -05:00
henrik karlsson
1c80dec3c0 [UBT]
* Changed so ParallelExecutor is creating build tasks as resources becomes available instead of building up a big task graph up front. The way the previous parallel executor was implemented could cause poor utilization of resources.

With this change plus 23915597  FortniteEditor wall time went down from ~15m to 13m  (tested three times with and without changes)

#preflight 63d5d8eb5c69f453c19e5385
#rb Joe.Kirchoff

[CL 23923458 by henrik karlsson in ue5-main branch]
2023-01-31 01:16:30 -05:00
Joe Kirchoff
042c3ca548 UnrealBuildTool: Use Normal priority for ParallelExecutor for asymmetric processors as BelowNormal can cause scheduling issues
#rnx
#rb trivial
#jira UE-168380
#preflight 63b8b7ce577437afe614728f

[CL 23605849 by Joe Kirchoff in ue5-main branch]
2023-01-06 19:29:19 -05:00
joe kirchoff
8046ba0c0d UnrealBuildTool: Ensure Action list is ordered before running any executor. Pass in IEnumerable instead of List
#rnx
#rb trivial
#preflight 635b20e0015611ae3a0d8f30

[CL 22831783 by joe kirchoff in ue5-main branch]
2022-10-28 13:27:22 -04:00
joe kirchoff
5c30a5e753 AutomationTool: Support running ParallelExecutor with all cores
#rnx
#rb trivial
#preflight 63361e532f447ea8dc2fce20

[CL 22271227 by joe kirchoff in ue5-main branch]
2022-09-30 11:12:48 -04:00
joe kirchoff
9d5e715748 UnrealBuildTool: Option to consider logical cores when running local builds
#jira UE-163438
#rnx
#rb none
#preflight 631baf2a304480f8f8bceac6

[CL 21930861 by joe kirchoff in ue5-main branch]
2022-09-09 19:16:18 -04:00
Ben Marsh
22feecaa2c UBT: Fix missing event markup when executing directly inside UBT. Event parsers are only currently run when using old logging methods.
#preflight none

[CL 20440822 by Ben Marsh in ue5-main branch]
2022-05-31 15:44:53 -04:00
Ben Marsh
cbb950c578 UBT: Use ILogger throughout UBT, rather than legacy EpicGames.Core.Log methods. All output going forwards should use structured logging rather than String.Format style logging.
#preflight 628e9dc6e746de4961f60032

[CL 20373777 by Ben Marsh in ue5-main branch]
2022-05-25 19:55:37 -04:00
Joe Kirchoff
930b065eba UnrealBuildTool: Expose ParallelExecutor.ProcessPriority to xml config
#rb none
#rnx
#preflight 628530752a7df8ed9fa70e54

[CL 20264894 by Joe Kirchoff in ue5-main branch]
2022-05-18 14:04:04 -04:00
Joe Kirchoff
c2ecac79bb UnrealBuildTool: Don't build a prereq action if it's not in the list of all actions to build
#jira UE-150951
#rb trivial
#rnx
#preflight 6272bbe02f6d177be3ccd558

[CL 20044734 by Joe Kirchoff in ue5-main branch]
2022-05-04 14:01:57 -04:00
jonathan adamczewski
11d542c545 UnrealBuildTool: add -CompactOutput command line option, supported by ParallelExecutor
Also can be set via BuildConfiguration.xml with:

<BuildConfiguration><bCompactOutput>true</bCompactOutput></BuildConfiguration>

#jira none
#preflight 626ae00e2c7b46848542a57f

[CL 19982217 by jonathan adamczewski in ue5-main branch]
2022-04-29 13:52:17 -04:00
Joe Kirchoff
a5d640b8b8 UnrealBuildTool: Retry failed HordeExecute remote tasks. Additional logging
#rnx
#rb none
#preflight 62606573dd47b4ad218101d5

[CL 19837598 by Joe Kirchoff in ue5-main branch]
2022-04-20 16:32:18 -04:00
Joe Kirchoff
58d330a938 UnrealBuildTool: Clean up ParallelExecutor to not require waiting manually on prereq actions by building a continuation tree of async tasks
Split logic into multiple static functions and make protected so it can be shared with an inherited class

#rnx
#rb Jonathan.Adamczewski
#preflight 62473274637925b5d3e0f1f3

[CL 19596364 by Joe Kirchoff in ue5-main branch]
2022-04-01 17:49:28 -04:00