#jira UE-199766
#rb zack.neyland
#tests tested on WSL, Ubuntu 22.04 Threadripper and Ubuntu 22.04 Megastation
[CL 30047912 by james singer in ue5-main branch]
This addresses issues with CPU resources on Mac going under utilized on systems where total RAM is only 2-3x the number of cores (e.g 8c/16GB, 10c/32GB) and the user has applications running (UE, XCode, browsers) that are occupying RAM.
Historically this check limited actions based on free memory to ensure action processes weren't forced to use swap, but there's strong evidence that limiting based on total system memory and relying on the OS to swap out other processes to make room is fine. That said only Mac has been extensively tested here in this scenario (and tends to have faster disk performance than other platforms). So for now we'll change Mac and leave Windows/Linux using the old behavior
Tests -
* Ran on an M1 8c/16GB Mac with Xcode & UE open and compared core utilization before (5) to after this change (8)
* Loaded up a bucket-load of apps, browser tabs, and a running game and verified engine built successfully 2x with no compiler OOMs
#jira UE-192238
#rb swarm
#swarm [at]Joe.Kirchoff [at]Zack.Neyland
[CL 26906094 by andrew grant in ue5-main branch]
The fix is to restore the <= 5.1 behavior of UBT where the MaxParallelActions argument specifies the maximum number of actions to execute in parallel, not a cap on UBTs heuristic-based selection.
This change also modifies BenchmarkBuild so it will default to the available processor count if no -cores argument is specified. This ensures the behavior of repeated runs will be deterministic as UBT may select different values if the system free memory changes
#jira UE-192236
#rb swarm
[REVIEW] [at]Joe.Kirchoff
#tests Ran UBT / BenchmarkBuild with arguments that forces the number of actions higher than the locally available core count
[CL 26889927 by andrew grant in ue5-main branch]
* Use object type rather than var
* Remove double newlines
* Use pattern matching
#rnx
#preflight 647780095d23eca37d28a387
[CL 25706751 by joe kirchoff in ue5-main branch]
[FYI] Joe.Kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: More automated code cleanup
#rnx
[CL 25695155 by joe kirchoff in ue5-main branch]
Fixed an issue where it was possible to do more parallel actions than cores available.
#rb joe.kirchoff
#preflight 640bc8aa363e9b40abeac263
[CL 24600925 by bryan sefcik in ue5-main branch]
Why do this:
Currently when compiling a cpp file with MSVC, it compiles across multiple cores while clang does not. This means that while we support limiting the number of cores(using ProcessorCountMultiplier), MSVC will use more cores than we specify. It also means that MSVC will always be faster when compiling because clang does not support compiling a cpp over multiple cores. To get similiar results when compiling with clang, we set the weight of MSVC to 1.5 and the weight of clang to 1.0. We then set the ProcessorCountMultiplier to 1.5. This results in MSVC and clang taking roughly the same amount of CPU utilization and clang compiles to be much faster.
Old Timing(secs) Old CPU Utilization New Timing New CPU Utilization(secs)
PlatformA AncientGame 590.94 51 431.47 73
MSVC AncientGameEditor 1016.96 94 1026.08 95
Clang AncientGameEditor 1543.72 63 1270.4 84
PlatformB AncientGame 494 52 396.95 74
Old = without weight path
New = with weight path
#jira
#rb christopher.waters, joe.kirchoff
#preflight 6409026c8832f48a4dc72025
[CL 24567859 by bryan sefcik in ue5-main branch]
* Optimization - WriteFileIfChanged taking enumerable of strings now join them all up to one string and use the WriteFileIfChanged path for one string. This saves some time seen when profiling
* Optimization - ExpandVariables use OrdinalCompare to find "$("
#preflight 63d838fcf626715201917046
#rb joe.kirchoff
[CL 23923964 by henrik karlsson in ue5-main branch]
