Commit Graph

135 Commits

Author SHA1 Message Date
zousar shaker
0832797d28 Ensure that Zen ProjectIds are unique for projects with the same name in differing roots (or in the same root) while attempting to maintain some easily recognizable desriptors for the project.
Staged builds are explicitly assigned a project id via the persistent UECommandline.txt file.  This works on mobile/consoles, but isn't implemented for Windows/Linux/Mac, so staged builds for those platforms don't have a persistent assignment of project id and instead rely on the launch by the editor including the appropriate commandlines.
Unstaged builds on Windows/Mac/Linux now determine project id independently after they have determined the project file path.  This happens after the project file has been determined and has had case corrected.  This ensures it is different/unique for multiple blueprint projects that share the same executable.
Fix a bug where second cook in editor to zen would stall because the queue thinks all additions are finished due to a boolean that isn't reset.

#rb devin.doucette
#rb per.larsson
#preflight 61b39a35a2562c8b1c40f81e

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 18432317 in //UE5/Release-5.0/... via CL 18435404
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18435895 by zousar shaker in ue5-release-engine-test branch]
2021-12-10 18:06:39 -05:00
devin doucette
27c1393427 CompressedBuffer: Removed partial decompression from FCompressedBuffer now that FCompressedBufferReader is available
Requiring the use of a separate reader type makes it more likely that readers will be reused, and makes it easier to audit reader usage going forward. Reusing readers is desirable to reduce the number of large temporary allocations made during partial decompression of a buffer.

- Added FCompressedBuffer::Save(FArchive&) and renamed FromCompressed(FArchive&) to Load(FArchive&).
- Added FCompressedBufferReaderSourceScope to set a buffer source within a scope.
- Added proper bounds checks to FNoneDecoder.
- Store the header checksum on the decoder context to allow raw blocks to be reused across sources.
- Decode the header on the fly to avoid a temporary header allocation when the header is in contiguous memory.

#rb Zousar.Shaker
#rnx
#preflight 61a98d53800738dbfbc84c73

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18382211 in //UE5/Release-5.0/... via CL 18382310
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v896-18170469)

[CL 18382377 by devin doucette in ue5-release-engine-test branch]
2021-12-06 10:16:05 -05:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00
matt peters
b9be0036df #jira UE-132874
#jira UE-132939
LargeMemoryArchive::ReleaseOwnership() sets TotalSize() to 0, so TotalSize must be called before calling ReleaseOwnership. Calling them both as arguments into a function was succeeding on some compilers and failing on others due to compiler optimization differences.
#rb Zousar.Shaker
#rnx
#[fyi] Francis.Hurteau

#ushell-cherrypick of 17947411 by Matt.Peters

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 18008712 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v885-17909292)

[CL 18008728 by matt peters in ue5-release-engine-test branch]
2021-11-01 17:20:22 -04:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00
robert millar
14fcaa7623 Add chunk id to output of iostore list command
#rb none
[FYI] carlmagnus.nordin

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 17738354 via CL 17738362 via CL 17738378 via CL 17738387 via CL 17738435
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v879-17706426)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17738662 by robert millar in ue5-release-engine-test branch]
2021-10-06 14:36:44 -04:00
robert millar
a43638b937 Modify 'List' command for investigating patch reordering issues.
Add LexToString for EIoChunkType.

#rb carlmagnus.nordin

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 17680962 in //UE5/Release-5.0/... via CL 17685338
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v875-17642767)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17685432 by robert millar in ue5-release-engine-test branch]
2021-09-30 19:54:44 -04:00
carlmagnus nordin
e6c7bce3ee Remove unused names from package headers, reducing the amount of data and number of names that have to be serialzed
* Strip names that are not used when cooking for UPS
* Strip names that are not used by the loader when staging for IoStore

#preflight 615424b7f4d2a4000107be68, 615450a875fae600010d4820
#rb pj.kack, francis.hurteau

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17674004 in //UE5/Release-5.0/... via CL 17674013
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v875-17642767)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17674014 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-30 02:05:41 -04:00
matt peters
763b1ad106 ZenStoreHttpClient: Expose connection failure to the Oplog owner to handle.
#rb Per.Larsson
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17661528 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v875-17642767)

[CL 17661553 by matt peters in ue5-release-engine-test branch]
2021-09-29 10:51:17 -04:00
per larsson
fa26f9c012 Make content-type/accept headers mandatory
#rb Stefan.Boberg
#rnx

#ROBOMERGE-AUTHOR: per.larsson
#ROBOMERGE-SOURCE: CL 17645289 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v875-17642767)

[CL 17645303 by per larsson in ue5-release-engine-test branch]
2021-09-28 09:53:06 -04:00
carlmagnus nordin
51c9794dea PackageStore refactor
* Decouple container concept from IoDispatcher
* Decoiuple PackageStore implementation from AsyncLoading2
* Restore ucas unmount fix that got kist when merrging from UE4
* Fix packages being left in the PackageStiore even after unmounting contaiiners

#rnx
#rb pj.kack, per.larsson
#preflight 61520cc52afc2d0001146ce7

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17641845 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v874-17637634)

[CL 17642353 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-28 04:00:33 -04:00
zousar shaker
96d0784963 Allow ZenServer to be autolaunched by editor use cases that require its presence without insisting that it always be autolaunched.
#rb matt.peters
#preflight 614b4150ee16e20001b146e3

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 17597162 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17597204 by zousar shaker in ue5-release-engine-test branch]
2021-09-22 13:00:17 -04:00
matt peters
e27fb01763 CookedPackageWriter iterative builds - change to use FAssetRegistryState as the list of previous files. Load the AssetRegistry from the AssetRegistry.bin file even when using ZenStore for the package storage.
#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17551332 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17551347 by matt peters in ue5-release-engine-test branch]
2021-09-17 09:32:48 -04:00
carlmagnus nordin
9f118fe8a4 Compile fixes for building UnrealPak in shipping config
#rnx
#rb pj.kack

#ROBOMERGE-OWNER: carlmagnus.nordin
#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17536937 via CL 17536972 via CL 17536975 via CL 17536976
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE-CONFLICT from-shelf
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17537001 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-16 07:48:26 -04:00
stefan boberg
e3eeb47216 Trivial logging fix: 'packges' -> 'packages'
#ROBOMERGE-AUTHOR: stefan.boberg
#ROBOMERGE-SOURCE: CL 17534852 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17534855 by stefan boberg in ue5-release-engine-test branch]
2021-09-16 03:18:06 -04:00
carlmagnus nordin
a627591ffa IoStore: Changes to allow IoStore to run as part of UnrealPak instead of as a cmdlet
#rb per.larsson
#rnx

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17517167 via CL 17517189 via CL 17517191 via CL 17519703
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17519714 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-15 10:39:23 -04:00
matt peters
54295195da CookOnTheFly - handle already-cooked packages from iterative builds.
#rb Per.Larsson
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17508341 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17508358 by matt peters in ue5-release-engine-test branch]
2021-09-14 15:15:17 -04:00
matt peters
208b83e781 DevelopmentAssetRegistry.bin: Reduce duplication by making a function to return this string.
#rb Per.Larsson
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17506665 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17506695 by matt peters in ue5-release-engine-test branch]
2021-09-14 13:31:59 -04:00
carlmagnus nordin
1cba69cba8 IoStore: Fixed issue that caused plugin remap redirects to be treated as localization redirects
#rb per.larsson
#rnx

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17501814 via CL 17502691 via CL 17502692 via CL 17502694
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17502695 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-14 06:08:53 -04:00
matt peters
bd9ec84b79 ZenStoreWriter: Make oplog keys case insensitive, to remove the possiblity of two oplog entries for the same packagename.
#rb CarlMagnus.Nordin
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17458021 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17458033 by matt peters in ue5-release-engine-test branch]
2021-09-08 09:51:13 -04:00
matt peters
56274a7640 ZenStoreWriter - Make CommitPackage take an array of attachments; remove separate variables for TargetDomainDependencies and send them in as an attachment.
ZenStoreHttpClient - Add IsConnected.
#rb CarlMagnus.Nordin, Stefan.Boberg
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 17456396 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[bot1]: dev-enginemerge-test

[CL 17456510 by matt peters in ue5-release-engine-test branch]
2021-09-08 07:04:39 -04:00
zousar shaker
8c80e5a866 Change Compact Binary Package API to represent binary attachments as compressed buffers identified by their raw hash.
Change Compact Binary Package serialization for binary attachments, compressed binary attachments and objects.

#rb devin.doucette

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 17450533 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17450587 by zousar shaker in ue5-release-engine-test branch]
2021-09-07 18:07:00 -04:00
carlmagnus nordin
2d97f9667e IoStore: Added support for storing compressed chunks in DDC
#rb pj.kack
#rnx

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17444267 via CL 17444348 via CL 17444353 via CL 17444359
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17444366 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-07 10:11:43 -04:00