Commit Graph

25 Commits

Author SHA1 Message Date
devin doucette
b2a07ea03e DDC: Merge from UE5/Main
#preflight 6288ff678828ea88c8af7034
#preflight 628ab5d93246d5019db76ed2
#rb none
#rnx

#ROBOMERGE-OWNER: devin.doucette
#ROBOMERGE-AUTHOR: Devin.Doucette
#ROBOMERGE-SOURCE: CL 20353148 via CL 20353832 via CL 20353839
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v948-20297126)

[CL 20355348 by devin doucette in ue5-main branch]
2022-05-24 16:40:25 -04:00
CarlMagnus Nordin
210d67b30e Promoted Zen project id generation to FApp
#rb pj.kack
#preflight 628769032c34da686ac82442

[CL 20316798 by CarlMagnus Nordin in ue5-main branch]
2022-05-23 01:49:25 -04:00
Per Larsson
ddbf9eb5d2 Zen - cook to custom project/oplog ID with -ZenStoreProject=<ProjectId> and -ZenStorePlatform=<OplogId>
#rb none
#rnx
#preflight 19755222

[CL 19804798 by Per Larsson in ue5-main branch]
2022-04-19 02:37:06 -04:00
matt peters
8b3fbdeede Fix bad auto-merge.
#rb None, Merge
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 19046456 via CL 19046457 via CL 19049399
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)

[CL 19065749 by matt peters in ue5-main branch]
2022-02-21 00:32:00 -05:00
matt peters
efc71c0005 Cooker: Disable experimental feature TryCollectKeyAndDependencies; it is only useful when other experimental feature EditorDomain is enabled and it costs 5 minutes in cook by the book.
#rb None, PortedFromUE5
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 19046327 via CL 19046336 via CL 19046345 via CL 19049398
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)

[CL 19065748 by matt peters in ue5-main branch]
2022-02-21 00:31:53 -05:00
devin doucette
cdedb37d66 DDC: Converted the Build API to UTF-8 where appropriate
#jira UE-133382
#rb Zousar.Shaker
#rnx
#preflight 61e798ee3f00a0a23ef78c49

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18655307 in //UE5/Release-5.0/... via CL 18655320 via CL 18655322
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v900-18638592)

[CL 18655349 by devin doucette in ue5-main branch]
2022-01-19 00:27:48 -05:00
zousar shaker
0832797d28 Ensure that Zen ProjectIds are unique for projects with the same name in differing roots (or in the same root) while attempting to maintain some easily recognizable desriptors for the project.
Staged builds are explicitly assigned a project id via the persistent UECommandline.txt file.  This works on mobile/consoles, but isn't implemented for Windows/Linux/Mac, so staged builds for those platforms don't have a persistent assignment of project id and instead rely on the launch by the editor including the appropriate commandlines.
Unstaged builds on Windows/Mac/Linux now determine project id independently after they have determined the project file path.  This happens after the project file has been determined and has had case corrected.  This ensures it is different/unique for multiple blueprint projects that share the same executable.
Fix a bug where second cook in editor to zen would stall because the queue thinks all additions are finished due to a boolean that isn't reset.

#rb devin.doucette
#rb per.larsson
#preflight 61b39a35a2562c8b1c40f81e

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 18432317 in //UE5/Release-5.0/... via CL 18435404
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18435895 by zousar shaker in ue5-release-engine-test branch]
2021-12-10 18:06:39 -05:00
matt peters
8d87746df1 EditorDomain: Use the EditorDomain's cache of EditorDomainHash instead of recalculating it every time it is needed for BulkData and TargetDomain keys.
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 18396652 in //UE5/Release-5.0/... via CL 18396659
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v896-18170469)

[CL 18396670 by matt peters in ue5-release-engine-test branch]
2021-12-07 13:12:55 -05:00
matt peters
2e44a269a3 EditorDomain: Improve performance of AppendDigest:
1) Change hash algorithm from FCbWriter (which collects memory in an array) to FBlake3 (which only keeps the memory necessary for the digest).
2) Change deduplication of Guids to deduplicate integer handles to the guids (cuts the deduplication time in half).
#rb Danny.Couture
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 18309477 in //UE5/Release-5.0/... via CL 18309534
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18309599 by matt peters in ue5-release-engine-test branch]
2021-11-29 10:08:16 -05:00
matt peters
a4a710baa7 TargetDomain::FetchCookAttachments: Remove string allocations
#rb None, Trivial
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 18305120 in //UE5/Release-5.0/... via CL 18305124
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18305132 by matt peters in ue5-release-engine-test branch]
2021-11-28 10:51:38 -05:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00
matt peters
763b1ad106 ZenStoreHttpClient: Expose connection failure to the Oplog owner to handle.
#rb Per.Larsson
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17661528 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v875-17642767)

[CL 17661553 by matt peters in ue5-release-engine-test branch]
2021-09-29 10:51:17 -04:00
zousar shaker
96d0784963 Allow ZenServer to be autolaunched by editor use cases that require its presence without insisting that it always be autolaunched.
#rb matt.peters
#preflight 614b4150ee16e20001b146e3

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 17597162 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17597204 by zousar shaker in ue5-release-engine-test branch]
2021-09-22 13:00:17 -04:00
matt peters
ead28b557b Load/Save BuildDefinitionList to the TargetDomain and EditorDomain.
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17458074 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17458084 by matt peters in ue5-release-engine-test branch]
2021-09-08 10:01:09 -04:00
matt peters
56274a7640 ZenStoreWriter - Make CommitPackage take an array of attachments; remove separate variables for TargetDomainDependencies and send them in as an attachment.
ZenStoreHttpClient - Add IsConnected.
#rb CarlMagnus.Nordin, Stefan.Boberg
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17456396 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[bot1]: dev-enginemerge-test

[CL 17456510 by matt peters in ue5-release-engine-test branch]
2021-09-08 07:04:39 -04:00
matt peters
79b61a9d91 Cooker: Hybrid Iterative cooks
Add the concept of pre-exploration of dependencies - searching assetregistry or target domain dependencies of the set of requested packages up front, to find all packages that will be cooked and allow us to do asynchronous and batch operations on them.
Convert FRequestCluster into an FPackageData container, holding packages in a substate of EPackageState::Request. This allows us to create a new RequestClusters when we encounter a package that was not found in the initial discovery.
Add hybriditerative mode for iterative cooks; this uses targetdomain keys to check for packages that exist from the previous cook, rather tracing the dependency graph to find all packages that transitively reference packages with modified guids.
Modify AssetRegistryGenerator to use a TMap of iteratively loaded packages that can be edied as packages cook, rather than using a const previous state; this supports hybrid iterative where we do not calculate the list of iterative packages until we encounter them in a request cluster.

#ROBOMERGE-SOURCE: CL 17373295 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17373304 by matt peters in ue5-release-engine-test branch]
2021-08-31 14:20:48 -04:00
matt peters
dc0db90119 PackageStoreWriter: Split IPackageStoreWriter into IPackageWriter, IPackageCookWrite, IPackageStoreWriter.
We plan to implement an IPackageWriter for workspace domain editor files to reduce duplication in SavePackage, and implementations for that purpose do not need to handle cook functionality.
IOStore-specific functionality is not needed by the cooker, and is pushed into IPackageStoreWriter.
#rb Per.Larsson, CarlMagnus.Nordin
#rnx

#ROBOMERGE-SOURCE: CL 17247188 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17247191 by matt peters in ue5-release-engine-test branch]
2021-08-20 10:02:58 -04:00
matt peters
7db8b0729a EditorDomain: Change whitelist/blacklist to new terms allowlist/blocklist.
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17233751 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17233770 by matt peters in ue5-release-engine-test branch]
2021-08-19 10:17:00 -04:00
matt peters
1cc9099110 Add TargetDomain.IterativeClassBlacklist and IterativeClassWhitelist.
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17185319 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17185369 by matt peters in ue5-release-engine-test branch]
2021-08-16 21:16:03 -04:00
matt peters
65926948cb Cooking: Create TargetDomain dependencies and key from the PackageBuildDependencyTracker and the AssetRegistry.
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17175833 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17175851 by matt peters in ue5-release-engine-test branch]
2021-08-16 10:13:06 -04:00
matt peters
f9a7f8d352 TargetDomain: Fix crash when TargetDomain is enabled caused by missing EndObject.
#rb None, Trivial
#rnx

#ROBOMERGE-SOURCE: CL 17115056 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17115060 by matt peters in ue5-release-engine-test branch]
2021-08-10 09:04:09 -04:00
matt peters
e20abf789a TargetDomain: Move TargetDomain utils implementation into a new cpp file. Rename GetDependenciesObject to CollectDependenciesObject to contrast with FetchDependencies. Create stubs for the fetch functions.
#rb None, trivial
#rnx

#ROBOMERGE-SOURCE: CL 17106809 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17106850 by matt peters in ue5-release-engine-test branch]
2021-08-09 16:08:19 -04:00