Commit Graph

102 Commits

Author SHA1 Message Date
Ben Marsh
e141cb6196 More UAT structured logging fixes.
#preflight 6408dee3b0544ef0b4bca696

[CL 24564590 by Ben Marsh in ue5-main branch]
2023-03-08 14:32:15 -05:00
Ben Marsh
235eb77726 Change to structured logging in other AutomationTool projects.
#preflight 6408c4478c0039bbf750e861

[CL 24562553 by Ben Marsh in ue5-main branch]
2023-03-08 12:43:35 -05:00
Ben Marsh
89d6613573 UAT: Refactor AutomationUtils and BuildGraph to use structured logging.
#preflight 6407c4c26e1dadfcdab25849

[CL 24553706 by Ben Marsh in ue5-main branch]
2023-03-07 21:23:47 -05:00
josh adams
bb86736ace - Fixed targets with names like Foo_Mac to not think they are platform extensions if they aren't under a Mac directory (platform extensions will be named with Platform name always)
#jira UE-178334
#rb David.Harvey
#preflight 640013dcdf66ed5fc1367d6a

[CL 24485971 by josh adams in ue5-main branch]
2023-03-02 13:21:06 -05:00
Joe Kirchoff
a46c2bb6c1 UnrealBuildTool: Reference any Target.cs files as rules dependencies so modifying them will update the Makefile, as these TargetRules can be referenced
#rnx
#rb Tim.Smith
#jira UE-168481
#preflight 63d7fa38d21dbe1d29525dd3

[CL 23910603 by Joe Kirchoff in ue5-main branch]
2023-01-30 12:31:04 -05:00
josh adams
1610c3bee3 UnrealArch/UnrealArchitectures changes
- Creates the UnrealArchitectures class, which wraps a list of UnrealArch objects
 - UnrealArch is a single architecture, expandable enum-like struct
 - There is no more concept of "no/default architecture", there is always a valid active architecture when building
 - Most uses of "string Architecture" are replaced with one of the two above, depending if multiple architectures are supported or not
 - UnrealArch has some platform-extensions for platform-specific naming (like Linux adds in LinuxName that turns, for instance, Arm64 -> aarch64-unknown-linux-gnueabi, which is used in folder names, etc)
 - UnrealArch has bIsX64 which can be used determine intel instruction set (as opposed to arm)
 - TargetRules class has an "Architecture" accessor that will return a single architecture if the active architectures is a single architecture, or throw an exception if multiple. This is useful in a majority of the cases where a paltform can only have a single architecture active in TargetRules (microsoft platforms, for instance, will create separate targets when compiling multiple architectures at once)
 - Added UnrealArchitectureConfig class, which contains all the architecture information for a platform (what architectures are supported, what ones are currently active for given project, etc)

#preflight 63c81fb5b065224750a1759e
#rb mike.fricker,roman.dzieciol,joe.kirchoff,dmytro.vovk,brandon.schaefer [various parts]
#p4v-preflight-copy 23562471

[CL 23829977 by josh adams in ue5-main branch]
2023-01-24 09:30:28 -05:00
Josh Adams
5b2f23198f - Allow staging/packaging of programs (requires the program have a basic .uproject, see the new SlateUGS.uproject)
- Added code to patch paths for handling Programs having source in /Engine/Source/Programs/XYZ, their config fies, etc in /Engine/Programs/XYZ, and staging using /XYZ
- Mac can now apply -specifiedarchitecture to UAT builds of programs
- Added support to SlateUGS to load Pak files (programs have to opt-in to it in their main Build.cs)
#rb brandon.schaefer,david.harvey
#preflight 631a03202b7fe03eb6b0f16a

[CL 21894674 by Josh Adams in ue5-main branch]
2022-09-08 11:22:00 -04:00
Wojciech Krywult
af488b1326 Automation: Fixed PropertiesCache cache with ProjectProperties that could add multiple values for the same key.
GetProjectProperties uses PropertiesCache collection (hashmap) to store project properties indexed by PropertyCacheKey struct (.uproject path + platforms + configurations). However, it didn't work as expected because PropertyCacheKey has array members that are reference types. Without a custom hashing implemented, this key would generate different hash codes even if elements stored in the arrays were the same. As a result, querying the same properties multiple times would generate multiple entries in the cache e.g.

GetProjectProperties("foo.uproject", {UnrealTargetPlatform.PC}, {UnrealTargetConfiguration.Debug})
GetProjectProperties("foo.uproject", {UnrealTargetPlatform.PC}, {UnrealTargetConfiguration.Debug})

would create two entries for PC/Debug.

Resolved by implementing the full set of Equals, GetHashCode and comparison operators.

#preflight 630e8d14556fc14dce7d11c5
#rb David.Harvey
#jira UE-162001
#rnx

[CL 21720344 by Wojciech Krywult in ue5-main branch]
2022-08-31 10:07:21 -04:00
Wojciech Krywult
abb6e2c33a Automation: Fixed staging of blueprint-only projects on some platforms.
The problem was that blueprint-only projects would return true for IsCodeBasedUProjectFile during staging. The issue was triggered by CL 21214930 where PropertyCacheKey (ProjectUtils.cs) was added. Before this change GetProjectProperties worked based only on the path to .uproject. After the change, it also takes the platform and configurations into account. It means that:

GetProjectProperties("...", SomePlatform, Debug)
GetProjectProperties("...", null, Debug)

may return different results. While it's probably the core or the fix this change implemented, it may given unexpected results because when null is passed as a list of platforms, internally all available platforms are taken into account and if only one of them requires the project to be compiled IsCodeBasedUProjectFile will return true, which may not be what our code is expected to see. That's what happened during staging - IsCodeBasedUProjectFile was invoked two times in a slightly different way leading to the automation code handling blueprint-only projects in an inconsistent-way during the whole process.

Fixed by modifying DeploymentContext to initialize IsCodeBasedProject by passing both the current platform and configurations.

#preflight 630e6bd0501b64ba334c3c2a
#rb none
#jira UE-162001
#rnx

[CL 21711241 by Wojciech Krywult in ue5-main branch]
2022-08-30 18:18:06 -04:00
andrew grant
499f555538 Restore 21213169
Improvements to BenchmarkBuild script.
- Added -editor-startup and -editor-game options to measure startup and startup time with -game
- Renamed -pie to -editor-pie to match other editor options-
- Allow different editor tests to use different sets of maps via -editor-startup=map2+map2, -editor-pie=map1+map2 etc
- Added AllEditor option to run all editor benchmarking
- Added -coldddc-noshared DDC option
- BenchmarkBuild now defaults to cleaning each targets after benchmarking its build time to save disk space (disable with -nopostclean)
- Moved Editor benchmark tests into separate files and reduced some duplicated code
- Updated help info

Other
- Fixed UAT GetProjectProperties returning inaccurate values if called twice with different platforms
- Doubled number of UBT logs that can be generated from a single UAT script
- Allow scripts to call UnrealBuild.CleanWithUBT to perform cleanup
- Simplified a lot of code

#rb na

#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21213895 via CL 21213903 via CL 21213910 via CL 21214414 via CL 21214472
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21214930 by andrew grant in ue5-main branch]
2022-07-22 00:56:39 -04:00
brandon boswell
338ca5c885 Back out changelist 21213169
#ROBOMERGE-AUTHOR: brandon.boswell
#ROBOMERGE-SOURCE: CL 21213801 via CL 21213879 via CL 21213889 via CL 21214176 via CL 21214241
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21214665 by brandon boswell in ue5-main branch]
2022-07-22 00:13:12 -04:00
andrew grant
2a13ce4d7c Improvements to BenchmarkBuild script.
- Added -editor-startup and -editor-game options to measure startup and startup time with -game
- Renamed -pie to -editor-pie to match other editor options
- Allow different editor tests to use different sets of maps via -editor-startup=map2+map2, -editor-pie=map1+map2 etc
- Added AllEditor option to run all editor benchmarking
- Added -coldddc-noshared DDC option
- BenchmarkBuild now defaults to cleaning each targets after benchmarking its build time to save disk space (disable with -nopostclean)
- Moved Editor benchmark tests into separate files and reduced some duplicated code
- Updated help info

Other

- Fixed UAT GetProjectProperties returning inaccurate values if called twice with different platforms
- Doubled number of UBT logs that can be generated from a single UAT script
- Allow scripts to call UnrealBuild.CleanWithUBT to perform cleanup
- Simplified a lot of code

#rb na

#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21213169 via CL 21213174 via CL 21213179 via CL 21214155 via CL 21214226
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21214663 by andrew grant in ue5-main branch]
2022-07-22 00:12:59 -04:00
Ben Marsh
cf7582e557 UAT: Fix exception when creating dummy targets in UAT.
#preflight none

[CL 20374333 by Ben Marsh in ue5-main branch]
2022-05-25 20:48:43 -04:00
Ben Marsh
cbb950c578 UBT: Use ILogger throughout UBT, rather than legacy EpicGames.Core.Log methods. All output going forwards should use structured logging rather than String.Format style logging.
#preflight 628e9dc6e746de4961f60032

[CL 20373777 by Ben Marsh in ue5-main branch]
2022-05-25 19:55:37 -04:00
zousar shaker
0832797d28 Ensure that Zen ProjectIds are unique for projects with the same name in differing roots (or in the same root) while attempting to maintain some easily recognizable desriptors for the project.
Staged builds are explicitly assigned a project id via the persistent UECommandline.txt file.  This works on mobile/consoles, but isn't implemented for Windows/Linux/Mac, so staged builds for those platforms don't have a persistent assignment of project id and instead rely on the launch by the editor including the appropriate commandlines.
Unstaged builds on Windows/Mac/Linux now determine project id independently after they have determined the project file path.  This happens after the project file has been determined and has had case corrected.  This ensures it is different/unique for multiple blueprint projects that share the same executable.
Fix a bug where second cook in editor to zen would stall because the queue thinks all additions are finished due to a boolean that isn't reset.

#rb devin.doucette
#rb per.larsson
#preflight 61b39a35a2562c8b1c40f81e

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 18432317 in //UE5/Release-5.0/... via CL 18435404
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18435895 by zousar shaker in ue5-release-engine-test branch]
2021-12-10 18:06:39 -05:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
jonathan adamczewski
b334e50489 Gauntlet: fix project path when launching on Mac
#jira UE-127192
#rb geoff.evans

#ROBOMERGE-AUTHOR: jonathan.adamczewski
#ROBOMERGE-SOURCE: CL 17590867 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17590877 by jonathan adamczewski in ue5-release-engine-test branch]
2021-09-21 21:15:35 -04:00
geoff evans
3e0b0fbc35 AutomationTool: Support for Editor in Project BuildEnvironments
This adds support for running Editor in BuildGraph Command and Commandlet tasks where the Project at hand has it's own BuildEnvironment.

#jira none
#rb ben.marsh
#preflight 6143be8d9bba9a0001dfc9ee

#ROBOMERGE-AUTHOR: geoff.evans
#ROBOMERGE-SOURCE: CL 17546102 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17546103 by geoff evans in ue5-release-engine-test branch]
2021-09-16 19:20:37 -04:00
branden turner
8e2d76d410 #jira UE-121181
Making project file generation wrt to target files consistent for temporary targets ( used for BP-only projects that are either nativized or enable a non-default plugin ) and project file generation.  Temp targets generated in this way didn't previously set DefaultBuildSettings to v2.

#rb none
[FYI] Sebastian.Thomeczek
[FYI] Jonathan.Adamczewski

#ROBOMERGE-SOURCE: CL 17390240 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17390267 by branden turner in ue5-release-engine-test branch]
2021-09-01 14:12:41 -04:00
jonathan adamczewski
f270855eef AutomationTool: Compile script modules within the application
Add a layer of caching to avoid running msbuild as much as possible.

#jira UE-109181
#rb ben.marsh

#ROBOMERGE-SOURCE: CL 17102399 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17102408 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-09 10:39:35 -04:00
jonathan adamczewski
64c210d294 AutomationTool, BuildUtilities:
UnrealBuild -> Unreal for EngineDirectory, RootDirectory, IsEngineInstalled, UnrealBuildToolPath
Remove CommandUtils EngineDirectory, RootDirectory, IsEngineInstalled - use equvalents from UnrealBuildBase.Unreal

#jira none

#ROBOMERGE-SOURCE: CL 16648181 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v833-16641396)

[CL 16648203 by jonathan adamczewski in ue5-release-engine-test branch]
2021-06-11 18:21:35 -04:00
aurel cordonnier
25a11deeac Merge from Release-Engine-Staging @ 16579919
This represents UE4/Main @ 16579691 and Dev-PerfTest @ 16579576

[CL 16581170 by aurel cordonnier in ue5-release-engine-test branch]
2021-06-07 20:09:45 -04:00
Ben Marsh
ab5822fd36 Fix non-deterministic use of GetHashCode() in ProjectUtils.cs.
#rb none
#rnx

[CL 15214570 by Ben Marsh in ue5-main branch]
2021-01-26 22:40:11 -04:00
Ben Marsh
6f927647b1 Changing C# code over to using EpicGames.Core over DotNETCommon.
#rb none
#rnx

[CL 14962096 by Ben Marsh in ue5-main branch]
2020-12-21 23:07:37 -04:00
Joakim Lindqvist
e7039d3d35 UBT and UAT now use .NET Core instead of Framework and Mono. This means that we use the same runtime on Windows, Linux and Mac. Further benefits including newer C# features and a lot of intresting features for the future around AOT and Tiered compilation.
Some behavior changes:
Output paths - Both tools are now output to a subdirectory of Binaries/Dotnet, I believe most hardcoded paths have been fixed up but there may be tools that will fail because of this.
UAT Plugin Building - As .NET Core does not support AppDomain unloading, how we build the plugins has changed quite a bit, these are now built before UAT is started rather then by UAT itself. If you just start UAT via RunUAT.bat/sh this should just continue to work.

#rb ben.marsh

[CL 14834347 by Joakim Lindqvist in ue5-main branch]
2020-12-02 06:57:13 -04:00