Commit Graph

155 Commits

Author SHA1 Message Date
joe kirchoff
8046ba0c0d UnrealBuildTool: Ensure Action list is ordered before running any executor. Pass in IEnumerable instead of List
#rnx
#rb trivial
#preflight 635b20e0015611ae3a0d8f30

[CL 22831783 by joe kirchoff in ue5-main branch]
2022-10-28 13:27:22 -04:00
bryan sefcik
6245f9d961 Minor optimizations in UBT to improve time when creating the action graph.
#jira
#preflight 6335f77e2f447ea8dc26f758

[CL 22264273 by bryan sefcik in ue5-main branch]
2022-09-30 01:41:38 -04:00
joe kirchoff
abdd5ac8ca UnrealBuildTool: Don't throw entire action graph as an exception when there is a cycle
#rnx
#rb trivial
#preflight 63349be6ce6dba45926fb2d5

[CL 22240106 by joe kirchoff in ue5-main branch]
2022-09-28 22:11:23 -04:00
joe kirchoff
9d5e715748 UnrealBuildTool: Option to consider logical cores when running local builds
#jira UE-163438
#rnx
#rb none
#preflight 631baf2a304480f8f8bceac6

[CL 21930861 by joe kirchoff in ue5-main branch]
2022-09-09 19:16:18 -04:00
Joe Kirchoff
e7bed3a793 UnrealBuildTool: Report in the log when a dependency file is missing instead of printing an invalid timestamp
#rnx
#rb trivial
#preflight 62bb99da30036d0db9438dc8

[CL 20867012 by Joe Kirchoff in ue5-main branch]
2022-06-28 20:22:32 -04:00
Tim Smith
06cac684a2 Added .so and .dylib files as files to consider as library files to help reduce relinking on Mac and Linux.
#rb joe.kirchoff, ben.marsh
#preflight 62a72b392a70e8deaccf4aaa

[CL 20626926 by Tim Smith in ue5-main branch]
2022-06-13 10:32:12 -04:00
Ben Marsh
cbb950c578 UBT: Use ILogger throughout UBT, rather than legacy EpicGames.Core.Log methods. All output going forwards should use structured logging rather than String.Format style logging.
#preflight 628e9dc6e746de4961f60032

[CL 20373777 by Ben Marsh in ue5-main branch]
2022-05-25 19:55:37 -04:00
jonathan adamczewski
11d542c545 UnrealBuildTool: add -CompactOutput command line option, supported by ParallelExecutor
Also can be set via BuildConfiguration.xml with:

<BuildConfiguration><bCompactOutput>true</bCompactOutput></BuildConfiguration>

#jira none
#preflight 626ae00e2c7b46848542a57f

[CL 19982217 by jonathan adamczewski in ue5-main branch]
2022-04-29 13:52:17 -04:00
Joe Kirchoff
2adc4a0593 HordeExecutor Prototype
#rnx
#rb Jonathan.Adamczewski
#preflight 625d9c17d4124345879fddb9

[CL 19789711 by Joe Kirchoff in ue5-main branch]
2022-04-18 13:33:43 -04:00
jonathan adamczewski
c1bb6901bf Upgrade to .NET 6.0
#jira UE-119846
#preflight 624cbb2ecc0872e96b0d5181

[CL 19652696 by jonathan adamczewski in ue5-main branch]
2022-04-06 14:46:50 -04:00
jonathan adamczewski
5c68d1d9ef Revert //UE5/Main/... changelist 19531725 - return to NET Core 3.1
#preflight 62422fd0470aff98e946bfce
#jira UE-119846

[CL 19534572 by jonathan adamczewski in ue5-main branch]
2022-03-28 18:25:19 -04:00
jonathan adamczewski
18ac15cbb1 Upgrade to .NET 6.0
#jira UE-119846
#preflight 6241fc06470aff98e943c035

[CL 19531725 by jonathan adamczewski in ue5-main branch]
2022-03-28 15:30:56 -04:00
Joe Kirchoff
236f98fcc5 UnrealBuildTool: Don't use a remote distribution executor if the total number of actions is less than the number of max local parallel actions
#jira UE-146958
#rb trivial
#preflight 623bb3c688538cd45e263663

[CL 19489660 by Joe Kirchoff in ue5-main branch]
2022-03-23 20:14:29 -04:00
robert millar
778680c3fc Add XGE config bUnavailableIfInUse to allow falling back to a different executor if XGE is in use.
Add XGE config MinActions to avoid using XGE for small groups of tasks (e.g. single file compile).

#rb jonathan.adamczewski

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19413669 via CL 19415283 via CL 19428032 via CL 19428154
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)

[CL 19432532 by robert millar in ue5-main branch]
2022-03-18 02:29:51 -04:00
jeff newquist
7dbd77f49a SN-DBS executor improvements
- Prevent the unnecessary scanning of response files
- Prevent the unnecessary scanning and distribution of header files for other platforms
- Add bAllowOverVpn and VpnSubnets XmlConfigFile settings to allow restriction of distribution over VPN
- Utilize ProgressWriter to better mark up progress text as is done with other executor implementations
#jira none
#rb joe.kirchoff
#preflight none

[CL 19208610 by jeff newquist in ue5-main branch]
2022-03-01 17:39:49 -05:00
Joe Kirchoff
da53ed939e UnrealBuildTool: Replace ParallelExecutor with (no longer experimental) TaskExecutor
#rnx
#rb none
#preflight 61e71d2bb56c33b8ecf48659

[CL 18646501 by Joe Kirchoff in ue5-main branch]
2022-01-18 15:19:06 -05:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00
jonathan adamczewski
6510a9750e UnrealBuildTool: ActionGraph.cs clean up formatting, naming & some other minor changes
#jira none
#trivial

#ROBOMERGE-AUTHOR: jonathan.adamczewski
#ROBOMERGE-SOURCE: CL 17557368 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17557391 by jonathan adamczewski in ue5-release-engine-test branch]
2021-09-17 15:59:13 -04:00
jonathan adamczewski
eba03f6900 UnrealBuildTool: Split ActionGraph.IsActionOutdated() into two functions
IsActionGraph() is complicated: recursively evaluating a graph, modifying data at each node (ActionHistory.UpdateProducingAttributes) making it easy to introduce hard-to-see race conditions, and difficult to optimize.

This change splits the function into two parts:

IsIndividualActionOutdated(): finds nodes that are out of date, based on the node's own data - able to be run in parallel, with less cross-thread contention.

IsActionOutdatedDueToPrerequisites(): recursively traverse the graph, to determine if an action is out of date based on its prerequisites. Does not run in parallel, but is fast (processes ~6000 actions in 2-3ms).

For a clean, simple project editor build, total runtime is <100ms, compared to ~150ms previously.

#jira none
[FYI] ben.marsh

#ROBOMERGE-AUTHOR: jonathan.adamczewski
#ROBOMERGE-SOURCE: CL 17553054 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17553068 by jonathan adamczewski in ue5-release-engine-test branch]
2021-09-17 11:46:11 -04:00
joe kirchoff
de1e588e8b UnrealBuildTool: Clean up some intellisense suggestions, format documents, remove and sort usings, etc.
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 17059447 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v850-17047176)

[CL 17059463 by joe kirchoff in ue5-release-engine-test branch]
2021-08-04 16:50:01 -04:00
joe kirchoff
eb833bca6f UnrealBuildTool: Action Executor that uses async tasks instead of background threads (currently disabled)
#rb none

#ROBOMERGE-SOURCE: CL 16682139 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16682147 by joe kirchoff in ue5-release-engine-test branch]
2021-06-15 19:38:23 -04:00
jonathan adamczewski
64c210d294 AutomationTool, BuildUtilities:
UnrealBuild -> Unreal for EngineDirectory, RootDirectory, IsEngineInstalled, UnrealBuildToolPath
Remove CommandUtils EngineDirectory, RootDirectory, IsEngineInstalled - use equvalents from UnrealBuildBase.Unreal

#jira none

#ROBOMERGE-SOURCE: CL 16648181 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v833-16641396)

[CL 16648203 by jonathan adamczewski in ue5-release-engine-test branch]
2021-06-11 18:21:35 -04:00
joe kirchoff
f2b90ec505 UnrealBuildTool: Remove LocalExecutor and use ParallelExecutor for all host platforms
Update ManagedProcess to capture stderr if using C# Process objects

#jira UE-117670
#rb Ben.Marsh

#ROBOMERGE-SOURCE: CL 16647451 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v833-16641396)

[CL 16647455 by joe kirchoff in ue5-release-engine-test branch]
2021-06-11 17:19:29 -04:00
joe kirchoff
212ef0663f [Backout] - CL16642253
[FYI] Joe.Kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Remove LocalExecutor and use ParallelExecutor for all host platforms
Update ManagedProcess to capture stderr if using C# Process objects

#jira UE-117670
#rb Ben.Marsh
#preflight 60c2a1d3730f8a000185ae66
#preflight 60c2a5bab1a4a300012ea45d

#ROBOMERGE-SOURCE: CL 16642682 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v833-16641396)

[CL 16642692 by joe kirchoff in ue5-release-engine-test branch]
2021-06-11 12:14:43 -04:00