Commit Graph

141 Commits

Author SHA1 Message Date
jeff newquist
7dbd77f49a SN-DBS executor improvements
- Prevent the unnecessary scanning of response files
- Prevent the unnecessary scanning and distribution of header files for other platforms
- Add bAllowOverVpn and VpnSubnets XmlConfigFile settings to allow restriction of distribution over VPN
- Utilize ProgressWriter to better mark up progress text as is done with other executor implementations
#jira none
#rb joe.kirchoff
#preflight none

[CL 19208610 by jeff newquist in ue5-main branch]
2022-03-01 17:39:49 -05:00
Joe Kirchoff
da53ed939e UnrealBuildTool: Replace ParallelExecutor with (no longer experimental) TaskExecutor
#rnx
#rb none
#preflight 61e71d2bb56c33b8ecf48659

[CL 18646501 by Joe Kirchoff in ue5-main branch]
2022-01-18 15:19:06 -05:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00
jonathan adamczewski
6510a9750e UnrealBuildTool: ActionGraph.cs clean up formatting, naming & some other minor changes
#jira none
#trivial

#ROBOMERGE-AUTHOR: jonathan.adamczewski
#ROBOMERGE-SOURCE: CL 17557368 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17557391 by jonathan adamczewski in ue5-release-engine-test branch]
2021-09-17 15:59:13 -04:00
jonathan adamczewski
eba03f6900 UnrealBuildTool: Split ActionGraph.IsActionOutdated() into two functions
IsActionGraph() is complicated: recursively evaluating a graph, modifying data at each node (ActionHistory.UpdateProducingAttributes) making it easy to introduce hard-to-see race conditions, and difficult to optimize.

This change splits the function into two parts:

IsIndividualActionOutdated(): finds nodes that are out of date, based on the node's own data - able to be run in parallel, with less cross-thread contention.

IsActionOutdatedDueToPrerequisites(): recursively traverse the graph, to determine if an action is out of date based on its prerequisites. Does not run in parallel, but is fast (processes ~6000 actions in 2-3ms).

For a clean, simple project editor build, total runtime is <100ms, compared to ~150ms previously.

#jira none
[FYI] ben.marsh

#ROBOMERGE-AUTHOR: jonathan.adamczewski
#ROBOMERGE-SOURCE: CL 17553054 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17553068 by jonathan adamczewski in ue5-release-engine-test branch]
2021-09-17 11:46:11 -04:00
joe kirchoff
de1e588e8b UnrealBuildTool: Clean up some intellisense suggestions, format documents, remove and sort usings, etc.
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 17059447 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v850-17047176)

[CL 17059463 by joe kirchoff in ue5-release-engine-test branch]
2021-08-04 16:50:01 -04:00
joe kirchoff
eb833bca6f UnrealBuildTool: Action Executor that uses async tasks instead of background threads (currently disabled)
#rb none

#ROBOMERGE-SOURCE: CL 16682139 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16682147 by joe kirchoff in ue5-release-engine-test branch]
2021-06-15 19:38:23 -04:00
jonathan adamczewski
64c210d294 AutomationTool, BuildUtilities:
UnrealBuild -> Unreal for EngineDirectory, RootDirectory, IsEngineInstalled, UnrealBuildToolPath
Remove CommandUtils EngineDirectory, RootDirectory, IsEngineInstalled - use equvalents from UnrealBuildBase.Unreal

#jira none

#ROBOMERGE-SOURCE: CL 16648181 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v833-16641396)

[CL 16648203 by jonathan adamczewski in ue5-release-engine-test branch]
2021-06-11 18:21:35 -04:00
joe kirchoff
f2b90ec505 UnrealBuildTool: Remove LocalExecutor and use ParallelExecutor for all host platforms
Update ManagedProcess to capture stderr if using C# Process objects

#jira UE-117670
#rb Ben.Marsh

#ROBOMERGE-SOURCE: CL 16647451 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v833-16641396)

[CL 16647455 by joe kirchoff in ue5-release-engine-test branch]
2021-06-11 17:19:29 -04:00
joe kirchoff
212ef0663f [Backout] - CL16642253
[FYI] Joe.Kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Remove LocalExecutor and use ParallelExecutor for all host platforms
Update ManagedProcess to capture stderr if using C# Process objects

#jira UE-117670
#rb Ben.Marsh
#preflight 60c2a1d3730f8a000185ae66
#preflight 60c2a5bab1a4a300012ea45d

#ROBOMERGE-SOURCE: CL 16642682 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v833-16641396)

[CL 16642692 by joe kirchoff in ue5-release-engine-test branch]
2021-06-11 12:14:43 -04:00
joe kirchoff
5ea7bcf834 UnrealBuildTool: Remove LocalExecutor and use ParallelExecutor for all host platforms
Update ManagedProcess to capture stderr if using C# Process objects

#jira UE-117670
#rb Ben.Marsh
#preflight 60c2a1d3730f8a000185ae66
#preflight 60c2a5bab1a4a300012ea45d

#ROBOMERGE-SOURCE: CL 16642253 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v833-16641396)

[CL 16642263 by joe kirchoff in ue5-release-engine-test branch]
2021-06-11 11:41:47 -04:00
jonathan adamczewski
d58996b916 UnrealBuildTool: move some path constants into BuildUtilities
RootDirectory, EngineDirectory, UnrealBuildToolPath are now found in BuildUtilities' UnrealBuild namesapce.

The way these are computed has changed. Previously, it was assumed that the application is UnrealBuildTool, and paths were constructed relative to that assembly.

Now, the assumption is that the process is located under a "Engine/Build/DotNET" sub-path and paths are constructed relative to that.

#jira none

#ROBOMERGE-SOURCE: CL 16607440 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v828-16531559)

[CL 16607455 by jonathan adamczewski in ue5-release-engine-test branch]
2021-06-09 12:55:13 -04:00
jonathan adamczewski
a865a21fb0 Move FileItem and DirectoryItem into BuildUtilities
#jira none

#ROBOMERGE-SOURCE: CL 16596289 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v828-16531559)

[CL 16596323 by jonathan adamczewski in ue5-release-engine-test branch]
2021-06-08 19:06:03 -04:00
Joe Kirchoff
bf5c868045 UnrealBuildTool: Print detailed information when two Actions conflict
#jira UE-100056
#rb Ben.Marsh

[CL 16012091 by Joe Kirchoff in ue5-main branch]
2021-04-14 16:55:57 -04:00
Marc Audy
bf80889353 UE5/Release-Engine-Staging to UE5/Main
This represents UE4/Main up to CL# 14958402

[CL 15028197 by Marc Audy in ue5-main branch]
2021-01-08 19:56:07 -04:00
Ben Marsh
6f927647b1 Changing C# code over to using EpicGames.Core over DotNETCommon.
#rb none
#rnx

[CL 14962096 by Ben Marsh in ue5-main branch]
2020-12-21 23:07:37 -04:00
Ben Marsh
1e9946779e UBT: Fix up a few more nullable references.
#rb none
#rnx

[CL 14960616 by Ben Marsh in ue5-main branch]
2020-12-20 20:07:40 -04:00
Ben Marsh
68f5abeb09 UBT: Convert UnrealBuildTool to use nullable references.
#rb none
#rnx

[CL 14960546 by Ben Marsh in ue5-main branch]
2020-12-20 18:47:42 -04:00
Ben Marsh
7047e080af UBT: Rename IAction to IExternalAction, to distinguish it from actions which only exist within UBT.
#rb none
#rnx

[CL 14823490 by Ben Marsh in ue5-main branch]
2020-11-30 16:58:32 -04:00
Ben Marsh
24dc06981b UBT: Remove support for outputting entire action graph for a target. This is not going to be valid with C++20 modules, since we require dependencies discovered at build time to feed back into the action graph execution.
#rb none
#rnx

[CL 14814704 by Ben Marsh in ue5-main branch]
2020-11-25 21:24:19 -04:00
Ben Marsh
7bcee7e396 UBT: Rename QueuedAction to LinkedAction.
#rb none
#rnx

[CL 14812791 by Ben Marsh in ue5-main branch]
2020-11-25 10:13:28 -04:00
Ben Marsh
8d1f11d053 UBT: Start to move Windows specific compile functionality into the VCCompileAction class.
#rb none
#rnx

[CL 14810985 by Ben Marsh in ue5-main branch]
2020-11-24 17:48:38 -04:00
Ben Marsh
f25f409ced UBT: Change the produced/prerequisite item lists for actions to be IEnumerables, so we can generate them dynamically using iterators.
#rb none
#rnx

[CL 14810772 by Ben Marsh in ue5-main branch]
2020-11-24 16:08:50 -04:00
Ben Marsh
29bfdd417f UBT: Allow actions to implement custom serialization methods, and store abstract interfaces for actions in the makefile.
#rb none
#rnx

[CL 14810441 by Ben Marsh in ue5-main branch]
2020-11-24 14:37:31 -04:00