* Added actions first produced item directory path as part of sorting for scheduling. This groups modules together nicely which will benefit when running things remotely where remotes takes batches of actions at the time meaning there is a higher chance each remote helper will need to download fewer variations of pch files
#rb joe.kirchoff
[CL 26856192 by henrik karlsson in ue5-main branch]
* Added AdditionalDescription parameter to ParallelExecutor.RunAction
* Added bCanExecuteInBox for actions
[CL 25968700 by henrik karlsson in ue5-main branch]
2) Added support in artifacts for relative paths
3) Modified ISPC header generation to be cached
At this time, only MSVC compiles are cached and -deterministic must be specified when building.
#rb josh.adams
#preflight 6470a172054af2576a2bd4b9
[CL 25647092 by tim smith in ue5-main branch]
* Added bCanExecuteRemotelyWithXGE so actions that can't execute remotely because of xge don't turn of remote execution for sndbs etc
#preflight 6448200b877716c878468070
#rb joe.kirchoff
[CL 25192137 by henrik karlsson in ue5-main branch]
* Added bCanCache to actions and enabled bCanCache to vc compile/link if /experimental:deterministic is set (because it can actually be cached then :-))
#preflight 6412214a924023147db1d829
#rb joe.kirchoff
[CL 24665382 by henrik karlsson in ue5-main branch]
Why do this:
Currently when compiling a cpp file with MSVC, it compiles across multiple cores while clang does not. This means that while we support limiting the number of cores(using ProcessorCountMultiplier), MSVC will use more cores than we specify. It also means that MSVC will always be faster when compiling because clang does not support compiling a cpp over multiple cores. To get similiar results when compiling with clang, we set the weight of MSVC to 1.5 and the weight of clang to 1.0. We then set the ProcessorCountMultiplier to 1.5. This results in MSVC and clang taking roughly the same amount of CPU utilization and clang compiles to be much faster.
Old Timing(secs) Old CPU Utilization New Timing New CPU Utilization(secs)
PlatformA AncientGame 590.94 51 431.47 73
MSVC AncientGameEditor 1016.96 94 1026.08 95
Clang AncientGameEditor 1543.72 63 1270.4 84
PlatformB AncientGame 494 52 396.95 74
Old = without weight path
New = with weight path
#jira
#rb christopher.waters, joe.kirchoff
#preflight 6409026c8832f48a4dc72025
[CL 24567859 by bryan sefcik in ue5-main branch]
Changed how single files are built. We don't want to invalidate makefile everytime we build single files since it destroys turnaround times. With this change a single file compile can take 2s (+ the actual compile time).
The new behavior injects a special action per module when creating the makefile. These actions can be used to on-the-fly create a proper compile action that follows the rules of the module that the specific file belongs to.. In a normal build these actions are ignored since the logic deciding which actions to build is backtraced from which binaries we want to create.
When a specific file compile is triggered, the logic deciding which files to build search up all these special actions and create a lookup based on which folders the special actions handle. It then try to find the special action that handles the specific file. The matching special action then creates a compile action that can handle that specific file and then queue up the action for execution. If no special action is found it falls back to try to use actions that have this specific file as input (ispc files for example)
Details:
* Removed lots of custom code for "specific files" handling
* Changed so pch (both private and shared) always use definition file. Added #pragma once and change so pch wrapper file include definition file. This made the adaptive path and specific file easy to implement (just disable pch in compile environment and it will just work)
* Added SingleFileAction for both VCToolChain and ClangToolChain. It now works to compile specific headers and cpp files. (It creates wrapper files on the fly to be able to compile all header files (compiling headers directly blow up if there are circular includes)
* Fixed so GenerateClangDatabase mode works with new changes
* Moved the logic that makes sure all (directly) depending cpp files are recompiled when .h are included in the singlefile option
#preflight 63dcc46f78716a01e8069649
#rb joe.kirchoff
[CL 24094027 by henrik karlsson in ue5-main branch]
* Changed how sorting of Actions is made. This to increase throughput of builds and reduce bubbles.
1. Changed so after actions are sorted based on dependency count we do a second pass and try to move linking to as early as possible. Reason is because linking is less cpu intense and makes sense to do in parallel with compiling instead of all linking happening at the end. Linking is also not remotely executed in some scenarios and therefore we want the local host to do the linking while other actions are remotely executed
2. Added a high priority flag that can explicitly be set on actions. This is now set on actions building precompiled headers which often takes a long time and can push out builds if they are only prioritized based on dependency count. This flag is also used in IWYU build mode to build headers first (to flush out errors early) and will most likely be used in header unit builds to prioritize header unit files before other code files.
With these changes plus a following change in ParallelExecutor.cs FortniteEditor wall time went down from ~15m to 13m (tested three times with and without changes)
#preflight 63d822b95c69f453c11c0ce5
#rb joe.kirchoff
[CL 23923446 by henrik karlsson in ue5-main branch]
Requires implementing IComparable\IEquatable in FileItem (and I also implemented them in DirectoryItem for consistency)
#rnx
#rb Tim.Smith
#preflight 63c1cce4a06ab115ea17ba4f
#preflight 63c70f4d7763631ba15486ea
[CL 23745743 by Joe Kirchoff in ue5-main branch]
