Commit Graph

109 Commits

Author SHA1 Message Date
Ben Marsh
d6d7c939e5 Copying //UE4/Dev-Build to Dev-Main (//UE4/Dev-Main)
#rb none

[CL 12726184 by Ben Marsh in Main branch]
2020-04-10 11:30:32 -04:00
andrew grant
d9b5eaf2ba Add Thread_UseAllCpuGroups to UnrealBuildTool config to enable UBT to use both CPU groups on high-core systems such as 64-core ThreadRippers.
See - https://docs.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/thread-useallcpugroups-element

Also added MaxParallelActions command-line flag to UBT that can be used to restrict the number of cores used for local (e.g. nonxge).

Added -cores=X+Y=Z option to benchmark script for profiling

#rb swarm
[at]ben.marsh
#jira nojira

#ROBOMERGE-SOURCE: CL 11845433 in //UE4/Release-4.25/... via CL 11845441
#ROBOMERGE-BOT: RELEASE (Release-4.25Plus -> Main) (v656-11643781)

[CL 11845479 by andrew grant in Main branch]
2020-03-02 15:50:41 -05:00
ben marsh
a26431c2ab UBT: Produce an error if any output path is longer than MAX_PATH on Windows.
#rb none
#jira UE-75445

#ROBOMERGE-SOURCE: CL 11638471 in //UE4/Release-4.25/... via CL 11638491
#ROBOMERGE-BOT: RELEASE (Release-4.25Plus -> Main) (v655-11596533)

[CL 11638504 by ben marsh in Main branch]
2020-02-26 13:44:50 -05:00
ben marsh
661248e268 UBT: Output a warning if any intermediate path is > 200 characters under the UE root directory.
#rb none
#jira UE-88446

#ROBOMERGE-SOURCE: CL 11624846 in //UE4/Release-4.25/... via CL 11624849
#ROBOMERGE-BOT: RELEASE (Release-4.25Plus -> Main) (v655-11596533)

[CL 11624868 by ben marsh in Main branch]
2020-02-25 20:49:06 -05:00
Ryan Durand
9ef3748747 Updating copyrights for Engine Programs.
#rnx
#rb none
#jira none

#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869242 in //Fortnite/Release-12.00/... via CL 10869536
#ROBOMERGE-BOT: FORTNITE (Main -> Dev-EngineMerge) (v613-10869866)

[CL 10870955 by Ryan Durand in Main branch]
2019-12-26 23:01:54 -05:00
michael trepka
011a101a70 UBT: Don't reference FASTBuild class in ActionGraph.cs if FASTBuild is unavailable
#rb Ben.Marsh


#ROBOMERGE-SOURCE: CL 10602281 via CL 10602282 via CL 10602283
#ROBOMERGE-BOT: (v607-10590470)

[CL 10602284 by michael trepka in Main branch]
2019-12-06 19:46:28 -05:00
michael trepka
48a2af4ef9 FASTBuild integration to allow distributed builds on Mac
[FYI] Chris.Bunner


#ROBOMERGE-SOURCE: CL 10600751 via CL 10600752 via CL 10600754
#ROBOMERGE-BOT: (v607-10590470)

[CL 10600755 by michael trepka in Main branch]
2019-12-06 17:10:57 -05:00
Michael Lentine
f9b3324b32 Copying //UE4/Dev-Physics to Dev-Main (//UE4/Dev-Main) @ 6903150
#rb none
#rnx

[CL 6903163 by Michael Lentine in Main branch]
2019-06-08 17:15:34 -04:00
Ben Marsh
0cc6e3dca6 Copying //UE4/Dev-Build to Dev-Main (//UE4/Dev-Main)
#rb none
#rnx

[CL 6631504 by Ben Marsh in Main branch]
2019-05-24 11:51:54 -04:00
ben marsh
4d9d6bc91f UBT: Store hashes in the action history rather than full command lines. Improves serialization performance by ~50ms.
#rb none
#jira UE-71116

#ROBOMERGE-SOURCE: CL 5337191 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)

[CL 5337944 by ben marsh in Main branch]
2019-03-07 11:45:15 -05:00
ben marsh
78e31eb6e1 UBT: Figure out outdated actions in parallel. Saves 0.5s on iterative compile.
#rb none
#jira
#rnx

#ROBOMERGE-SOURCE: CL 5315074 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)

[CL 5317566 by ben marsh in Main branch]
2019-03-06 14:52:27 -05:00
ben marsh
eeb6345cd1 UBT: Add additional timing info into logs.
#rb none
#rnx
#jira

#ROBOMERGE-SOURCE: CL 5312405 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)

[CL 5316167 by ben marsh in Main branch]
2019-03-06 13:30:40 -05:00
Ben Marsh
a22b952aa9 Copying //UE4/Dev-Build to Dev-Main (//UE4/Dev-Main)
#rb none
#rnx

[CL 4718806 by Ben Marsh in Main branch]
2019-01-14 12:11:24 -05:00
Ben Marsh
26b42416d4 UBT: Add a "hybrid" executor, which uses the XGE executor for large numbers of parallel actions, and the Parallel executor for any leaf actions that will not saturate the local machine. For platforms that take a long time to link, this frees up XGE to be used for compiling other branches.
To enable, set <BuildConfiguration><bEnableHybridExecutor>true</bEnableHybridExecutor></BuildConfiguration> in the BuildConfiguration.xml file.

#fyi Luke.Thatcher
#rb none

[CL 4709316 by Ben Marsh in Dev-Build branch]
2019-01-11 10:50:35 -05:00
Ben Marsh
e8ef63a8fd UBT: Only reset cached info for generated code directories after running UHT. Resetting cached info for all files causes slowdown for dependency checking.
#rb none
#rnx

[CL 4683841 by Ben Marsh in Dev-Build branch]
2019-01-05 20:25:15 -05:00
Ben Marsh
13cee21503 UBT: Fix another edge case for import library dependencies triggering new dependent items to be rebuilt.
#rb none
#rnx

[CL 4675997 by Ben Marsh in Dev-Build branch]
2019-01-02 09:30:02 -05:00
Ben Marsh
d1dcae8937 UBT: Fix import library changes being treated as outdated prerequisites in the action graph. Causes larger builds than necessary.
#rb none
#rnx

[CL 4675964 by Ben Marsh in Dev-Build branch]
2019-01-02 09:07:59 -05:00
Ben Marsh
ccde903b49 UBT: Use FileReference and DirectoryReference instead of raw strings in actions.
#rb none
#rnx

[CL 4674720 by Ben Marsh in Dev-Build branch]
2018-12-27 10:48:30 -05:00
Ben Marsh
703be91072 UBT: Add an exception for propagating error codes to the calling process, rather than having to pass them by return values. Also suppress "produced item not found" errors when the compile has already failed.
#rb none

[CL 4671996 by Ben Marsh in Dev-Build branch]
2018-12-20 15:10:04 -05:00
Ben Marsh
b4e5d0ee81 UBT: If UnrealHeaderTool needs rebuilding, do so from the same process rather than spawning a child UBT instance. This is significantly faster, since we're already fully initialized and have cached a lot of state.
#rb none

[CL 4671843 by Ben Marsh in Dev-Build branch]
2018-12-20 11:29:04 -05:00
Ben Marsh
c5b6fefcc7 UBT: Add a DirectoryItem class, similar to FileItem, which caches directory contents and properties. Allows constructing FileItem objects from enumerated FileInfo objects, avoiding additional I/O hit to determine attributes. DirectoryLookupCache is converted to use DirectoryItem objects as part of this change.
#rb none
#rnx

[CL 4670115 by Ben Marsh in Dev-Build branch]
2018-12-19 10:44:11 -05:00
Ben Marsh
34a4a0d74b UBT: Convert ActionGraph to a static class, and pass a list of actions through the toolchains.
#rb none
#rnx

[CL 4667670 by Ben Marsh in Dev-Build branch]
2018-12-17 23:48:27 -05:00
Ben Marsh
573d50dea5 UBT: Move handling of single file compiles into the build mode.
#rb none
#rnx

[CL 4667558 by Ben Marsh in Dev-Build branch]
2018-12-17 21:23:49 -05:00
Ben Marsh
f01ddb51ae UBT: Add support for building multiple targets in parallel.
* A makefile is now constructed for each target, and the outdated actions are combined into a single graph before execution.
* Makefiles are used in all build modes, regardless of whether they are saved to disk, and represent the vector by which a target's build actions are passed to the executor.
* The -JsonExport= argument has been removed, and a separate JsonExport mode has been added (invoke with eg. "-Mode=JsonExport UE4Editor Win64 Development")
* Multiple action history files are kept open at one time, and a hierarchy is used for each target.

#rb none

[CL 4667512 by Ben Marsh in Dev-Build branch]
2018-12-17 20:49:30 -05:00
Ben Marsh
0c9707d678 UBT: Use compiler-generated dependency information for source file dependencies. This ensures that the dependency list is accurate, and makes incremental builds more reliable.
* A standalone Win32 application called cl-filter is used to parse output from running the Visual Studio compiler with the /showIncludes option, and generate a dependency list similar to Clang.
* Actions which produce an external dependency list declare it explicitly. If the dependency list is missing, or any of the contents are out of date, the action is rebuilt.
* The contents of dependency lists are cached by the CppDependencyCache class for future iterations.

#rb none

[CL 4665060 by Ben Marsh in Dev-Build branch]
2018-12-16 14:22:47 -05:00