Fixed an issue where it was possible to do more parallel actions than cores available.
#rb joe.kirchoff
#preflight 640bc8aa363e9b40abeac263
[CL 24600925 by bryan sefcik in ue5-main branch]
Why do this:
Currently when compiling a cpp file with MSVC, it compiles across multiple cores while clang does not. This means that while we support limiting the number of cores(using ProcessorCountMultiplier), MSVC will use more cores than we specify. It also means that MSVC will always be faster when compiling because clang does not support compiling a cpp over multiple cores. To get similiar results when compiling with clang, we set the weight of MSVC to 1.5 and the weight of clang to 1.0. We then set the ProcessorCountMultiplier to 1.5. This results in MSVC and clang taking roughly the same amount of CPU utilization and clang compiles to be much faster.
Old Timing(secs) Old CPU Utilization New Timing New CPU Utilization(secs)
PlatformA AncientGame 590.94 51 431.47 73
MSVC AncientGameEditor 1016.96 94 1026.08 95
Clang AncientGameEditor 1543.72 63 1270.4 84
PlatformB AncientGame 494 52 396.95 74
Old = without weight path
New = with weight path
#jira
#rb christopher.waters, joe.kirchoff
#preflight 6409026c8832f48a4dc72025
[CL 24567859 by bryan sefcik in ue5-main branch]
* Changed so ParallelExecutor is creating build tasks as resources becomes available instead of building up a big task graph up front. The way the previous parallel executor was implemented could cause poor utilization of resources.
With this change plus 23915597 FortniteEditor wall time went down from ~15m to 13m (tested three times with and without changes)
#preflight 63d5d8eb5c69f453c19e5385
#rb Joe.Kirchoff
[CL 23923458 by henrik karlsson in ue5-main branch]
Also can be set via BuildConfiguration.xml with:
<BuildConfiguration><bCompactOutput>true</bCompactOutput></BuildConfiguration>
#jira none
#preflight 626ae00e2c7b46848542a57f
[CL 19982217 by jonathan adamczewski in ue5-main branch]
Split logic into multiple static functions and make protected so it can be shared with an inherited class
#rnx
#rb Jonathan.Adamczewski
#preflight 62473274637925b5d3e0f1f3
[CL 19596364 by Joe Kirchoff in ue5-main branch]
Disabled by default.
When enabled, adds wall time and CPU time numbers for each action e.g.
[543/665] (Wall: 6.33s CPU: 8.19s) Compile AssetBundleData.cpp
[544/665] (Wall: 3.77s CPU: 4.03s) Compile SavePackage2.cpp
[545/665] (Wall: 3.91s CPU: 4.56s) Compile WeakObjectPtr.cpp
#jira none
#rb joe.kirchoff
#ROBOMERGE-AUTHOR: jonathan.adamczewski
#ROBOMERGE-SOURCE: CL 17636393 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)
[CL 17636432 by jonathan adamczewski in ue5-release-engine-test branch]
Can be used to reduce the number of build actions that will be run in parallel by ParallelExecutor or TaskExecutor.
Example use, in MyProject.Target.cs:
public class MyProjectTarget : TargetRules
{
public MyProjectTarget(TargetInfo Target) : base(Target)
{
Type = TargetType.Game;
DefaultBuildSettings = BuildSettingsVersion.V2;
ExtraModuleNames.AddRange( new string[] { "MyProject" } );
MemoryPerActionGB = 4;
}
}
#jira none
#ROBOMERGE-AUTHOR: jonathan.adamczewski
#ROBOMERGE-SOURCE: CL 17546182 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)
[CL 17546186 by jonathan adamczewski in ue5-release-engine-test branch]
Process.StartTime raises an exception on non-windows platforms if the process has already exited, so cache it immediately after starting the FrameworkProcess
Process.TotalProcessorTime raises an exception on non-windows if the process has also exited, so don't do that
#rb none
#rnx
#ROBOMERGE-SOURCE: CL 16863633 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)
[CL 16863635 by joe kirchoff in ue5-release-engine-test branch]
[FYI] joe.kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Use process directly to get wall clock execution time
Process.StartTime raises an exception on non-windows platforms if the process has already exited, so cache it immediately after starting the FrameworkProcess
#rb none
#rnx
#ROBOMERGE-SOURCE: CL 16863370 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)
[CL 16863374 by joe kirchoff in ue5-release-engine-test branch]
