Commit Graph

8 Commits

Author SHA1 Message Date
Per Larsson
b20b928cf7 Editor Domain - fix fallouts after bulk data refactor
- allow file region of type None
- don't early out when saving bulk data by reference

#rb trivial
#jira none
#preflight 639b6de30a67152550853ab9
#rnx

[CL 23529938 by Per Larsson in ue5-main branch]
2022-12-15 14:35:36 -05:00
Per Larsson
2efa87995e Bulk Data - data resource table
This change aims to unify how BulkData and DerivedData references are stored and loaded from disk.
The structured part of Bulk Data i.e. Flags,Offset and Size has been moved out from the export data
and into a lookup table in the package header. This allows for bulk data to be rearranged w/o needing to recook but it will
also be used for derived data references to allow loading DDC data both from the runtime.
Currenlty this change ONLY effects cooked output.
Inline data can now be reloaded when loading from I/O store, all consistency checks
when discarding/reloading inline data has been removed.
When cooking, exports are serialized into it's own archive enabling the package header
to be modified after cooking the exports. This also makes it easer to rearrange the exports
in the export bundles for optimal loading performance, i.e. there is no longer a dependency
of the cooked serial offset from the begnning of the .uasset file.

#rb CarlMagnus.Nordi, PJ.Kack, Paul.Chipchase
#jira none
#preflight 637387db9e3bea807941a850
#rnx

[CL 23489396 by Per Larsson in ue5-main branch]
2022-12-13 03:51:37 -05:00
matt peters
daff9f8c5d EditorDomain: Implement support for PackageTrailers in the same package as BulkData in a regular BulkData section for PackageWriters, so that EditorDomain can have both references to BulkData in a WorkspaceDomain PackageTrailer and embedded BulkDatas that needed to be resaved.
Because the PackageFooter comes before the PackageTrailer, we need to change accounting for the PackageFooter in PackageWriters to include adjusting the VirtualExportsFileOffset that is passed into the PackageTrailer.
#rb Paul.Chipchase
#rnx
#preflight 638cbb1135192facc13f83e6

[CL 23386395 by matt peters in ue5-main branch]
2022-12-04 12:54:00 -05:00
Matt Peters
02ba0b5b70 ZenStoreWriter: Fix assertion "Trying to add package that is already pending" if the cooker commits a package a second time while the first commit of the package is queued for asynchronous completion aand has not yet reached CommitPackageInternal.
This is necessary because as of CL 20151503, the cooker can commit the same package multiple times if the first save failed due to a timeout.

#fyi CarlMagnus.Nordin
#rnx
#rb Per.Larsson
#preflight 62853ba90f0d5076dee74ced

[CL 20265738 by Matt Peters in ue5-main branch]
2022-05-18 14:49:11 -04:00
Dan Thompson
cd73707429 Update package writers to gather package hashes internally, then have CookByTheBookFinished copy them to asset package datas.
Add iostore chunk hashes to the asset registry during cook.
Remove old hashing code, which was extensive. This moves the async writes in to UE::Task from the old AsyncWorkSequence().
#rb Matt.Peters
#rb Jeff.Roberts
#preflight 624b24a5f73c316f68303946

[CL 19611812 by Dan Thompson in ue5-main branch]
2022-04-04 13:26:27 -04:00
francis hurteau
d262daac13 Added support to Loose File Package Writer to write multiple package output per input
Added similar support to the package store manifest
Zen Store support will be added separately
Added some extra data to linker object resources to be used by the iostore package optimizer to properly support optional package chunks.

Multiple cooking outputs is used to support "optional objects" which are editor data package exports that would be put in separate iostore containers than the normal cooked game content.

#jira UE-129801
#rb Matt.Peters, CarlMagnus.Nordin
#preflight 61e5d3bb873f2ea48f48166c

#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 18639742 in //UE5/Release-5.0/... via CL 18639751 via CL 18639758
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v900-18638592)

[CL 18639762 by francis hurteau in ue5-main branch]
2022-01-18 09:56:23 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00