Commit Graph

162 Commits

Author SHA1 Message Date
joe kirchoff
7aa214f6cb UnrealBuildTool: Remove obsolete HordeExecutor prototype
#rnx
[FYI] Ben.Marsh

[CL 24471319 by joe kirchoff in ue5-main branch]
2023-03-01 16:52:35 -05:00
Joe Kirchoff
6f904d0237 UnrealBuildTool: Simplify action conflict filename
#rnx
#rb trivial
#jira UE-176926
#preflight skipped

[CL 24214875 by Joe Kirchoff in ue5-main branch]
2023-02-14 12:37:45 -05:00
joe kirchoff
346eeabb72 UnrealBuildTool: When checking timestamps of produced files, also consider creation time as in some cases copied outputs (such as from a cache) can have a newer creation time than last write time
#rnx
#rb Ryan.Hummer
#preflight 63d9887dd21dbe1d290047ab

[CL 23942163 by joe kirchoff in ue5-main branch]
2023-01-31 19:26:51 -05:00
henrik karlsson
5f7fa70d36 [UBT]
* Changed how sorting of Actions is made. This to increase throughput of builds and reduce bubbles.
  1. Changed so after actions are sorted based on dependency count we do a second pass and try to move linking to as early as possible. Reason is because linking is less cpu intense and makes sense to do in parallel with compiling instead of all linking happening at the end. Linking is also not remotely executed in some scenarios and therefore we want the local host to do the linking while other actions are remotely executed
   2. Added a high priority flag that can explicitly be set on actions. This is now set on actions building precompiled headers which often takes a long time and can push out builds if they are only prioritized based on dependency count. This flag is also used in IWYU build mode to build headers first (to flush out errors early) and will most likely be used in header unit builds to prioritize header unit files before other code files.

With these changes plus a following change in ParallelExecutor.cs FortniteEditor wall time went down from ~15m to 13m  (tested three times with and without changes)

#preflight 63d822b95c69f453c11c0ce5
#rb joe.kirchoff

[CL 23923446 by henrik karlsson in ue5-main branch]
2023-01-31 01:16:13 -05:00
Josh Adams
11bb4c4c51 - Don't do the path length check (focused on Portability between users) for files in Intermediate directory, as these files are inherently not portable
#rb tim.smith
#preflight 63d03aeda605f887d1e08688

[CL 23837034 by Josh Adams in ue5-main branch]
2023-01-24 15:40:12 -05:00
Joe Kirchoff
e213963f90 UnrealBuildTool: Stabilize file path for action confilct
#rnx
#rb trivial
#preflight 63c1d69cb065224750e0c734

[CL 23687890 by Joe Kirchoff in ue5-main branch]
2023-01-13 17:12:52 -05:00
Joe Kirchoff
830db35fff UnrealBuildTool: Don't repeat conflict messages for the same two Actions if multiple produced items are conflicting
#rnx
#rb trivial
#preflight 63c0a2d22a6acaf16234c4e2

[CL 23672080 by Joe Kirchoff in ue5-main branch]
2023-01-12 19:20:52 -05:00
joe kirchoff
8046ba0c0d UnrealBuildTool: Ensure Action list is ordered before running any executor. Pass in IEnumerable instead of List
#rnx
#rb trivial
#preflight 635b20e0015611ae3a0d8f30

[CL 22831783 by joe kirchoff in ue5-main branch]
2022-10-28 13:27:22 -04:00
bryan sefcik
6245f9d961 Minor optimizations in UBT to improve time when creating the action graph.
#jira
#preflight 6335f77e2f447ea8dc26f758

[CL 22264273 by bryan sefcik in ue5-main branch]
2022-09-30 01:41:38 -04:00
joe kirchoff
abdd5ac8ca UnrealBuildTool: Don't throw entire action graph as an exception when there is a cycle
#rnx
#rb trivial
#preflight 63349be6ce6dba45926fb2d5

[CL 22240106 by joe kirchoff in ue5-main branch]
2022-09-28 22:11:23 -04:00
joe kirchoff
9d5e715748 UnrealBuildTool: Option to consider logical cores when running local builds
#jira UE-163438
#rnx
#rb none
#preflight 631baf2a304480f8f8bceac6

[CL 21930861 by joe kirchoff in ue5-main branch]
2022-09-09 19:16:18 -04:00
Joe Kirchoff
e7bed3a793 UnrealBuildTool: Report in the log when a dependency file is missing instead of printing an invalid timestamp
#rnx
#rb trivial
#preflight 62bb99da30036d0db9438dc8

[CL 20867012 by Joe Kirchoff in ue5-main branch]
2022-06-28 20:22:32 -04:00
Tim Smith
06cac684a2 Added .so and .dylib files as files to consider as library files to help reduce relinking on Mac and Linux.
#rb joe.kirchoff, ben.marsh
#preflight 62a72b392a70e8deaccf4aaa

[CL 20626926 by Tim Smith in ue5-main branch]
2022-06-13 10:32:12 -04:00
Ben Marsh
cbb950c578 UBT: Use ILogger throughout UBT, rather than legacy EpicGames.Core.Log methods. All output going forwards should use structured logging rather than String.Format style logging.
#preflight 628e9dc6e746de4961f60032

[CL 20373777 by Ben Marsh in ue5-main branch]
2022-05-25 19:55:37 -04:00
jonathan adamczewski
11d542c545 UnrealBuildTool: add -CompactOutput command line option, supported by ParallelExecutor
Also can be set via BuildConfiguration.xml with:

<BuildConfiguration><bCompactOutput>true</bCompactOutput></BuildConfiguration>

#jira none
#preflight 626ae00e2c7b46848542a57f

[CL 19982217 by jonathan adamczewski in ue5-main branch]
2022-04-29 13:52:17 -04:00
Joe Kirchoff
2adc4a0593 HordeExecutor Prototype
#rnx
#rb Jonathan.Adamczewski
#preflight 625d9c17d4124345879fddb9

[CL 19789711 by Joe Kirchoff in ue5-main branch]
2022-04-18 13:33:43 -04:00
jonathan adamczewski
c1bb6901bf Upgrade to .NET 6.0
#jira UE-119846
#preflight 624cbb2ecc0872e96b0d5181

[CL 19652696 by jonathan adamczewski in ue5-main branch]
2022-04-06 14:46:50 -04:00
jonathan adamczewski
5c68d1d9ef Revert //UE5/Main/... changelist 19531725 - return to NET Core 3.1
#preflight 62422fd0470aff98e946bfce
#jira UE-119846

[CL 19534572 by jonathan adamczewski in ue5-main branch]
2022-03-28 18:25:19 -04:00
jonathan adamczewski
18ac15cbb1 Upgrade to .NET 6.0
#jira UE-119846
#preflight 6241fc06470aff98e943c035

[CL 19531725 by jonathan adamczewski in ue5-main branch]
2022-03-28 15:30:56 -04:00
Joe Kirchoff
236f98fcc5 UnrealBuildTool: Don't use a remote distribution executor if the total number of actions is less than the number of max local parallel actions
#jira UE-146958
#rb trivial
#preflight 623bb3c688538cd45e263663

[CL 19489660 by Joe Kirchoff in ue5-main branch]
2022-03-23 20:14:29 -04:00
robert millar
778680c3fc Add XGE config bUnavailableIfInUse to allow falling back to a different executor if XGE is in use.
Add XGE config MinActions to avoid using XGE for small groups of tasks (e.g. single file compile).

#rb jonathan.adamczewski

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19413669 via CL 19415283 via CL 19428032 via CL 19428154
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)

[CL 19432532 by robert millar in ue5-main branch]
2022-03-18 02:29:51 -04:00
jeff newquist
7dbd77f49a SN-DBS executor improvements
- Prevent the unnecessary scanning of response files
- Prevent the unnecessary scanning and distribution of header files for other platforms
- Add bAllowOverVpn and VpnSubnets XmlConfigFile settings to allow restriction of distribution over VPN
- Utilize ProgressWriter to better mark up progress text as is done with other executor implementations
#jira none
#rb joe.kirchoff
#preflight none

[CL 19208610 by jeff newquist in ue5-main branch]
2022-03-01 17:39:49 -05:00
Joe Kirchoff
da53ed939e UnrealBuildTool: Replace ParallelExecutor with (no longer experimental) TaskExecutor
#rnx
#rb none
#preflight 61e71d2bb56c33b8ecf48659

[CL 18646501 by Joe Kirchoff in ue5-main branch]
2022-01-18 15:19:06 -05:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00