Commit Graph

953 Commits

Author SHA1 Message Date
CarlMagnus Nordin
df930de60e ZenLoader in editor: Remove garbage objects from linker export tables when notifying unreachable objects so that we don't end up with garbage objects in the import store
#rb pj.kack
#preflight 632ae9b19840225da2c50cc1

[CL 22109845 by CarlMagnus Nordin in ue5-main branch]
2022-09-21 06:47:50 -04:00
per larsson
4549f92468 Bulk Data - crash fix for bulk data with wrong flags getting uploaded to DDC
#rb Paul.Chipchase
#preflight none
#rnx

[CL 22088631 by per larsson in ue5-main branch]
2022-09-20 00:48:35 -04:00
carlmagnus nordin
9355c00328 EditorPackageLoader: Fixed issue with exports from already loaded uncooked packages not being registered in the import store
#rb pj.kack
#preflight 63284780ae4e65ae63c4da96

[CL 22084676 by carlmagnus nordin in ue5-main branch]
2022-09-19 19:49:50 -04:00
Matt Peters
97d33caf50 MeshDescription: In addition to applying the package's CustomVersions when deserializing MeshDescription out of BulkData, also apply the UEVersion and LicenseeUEVersion.
#jira UE-160855
#rb Richard.TalbotWatkin
#rnx
#preflight 63289253ea1c94f4c5e2fd67

[CL 22076022 by Matt Peters in ue5-main branch]
2022-09-19 13:01:00 -04:00
carlmagnus nordin
98fd341d01 ZenLoader: Let the async loading thread progress on other packages if there's no more work to do for the current sync load
#preflight 63245c4bae4e65ae63e96d20
#rb pj.kack

[CL 22059800 by carlmagnus nordin in ue5-main branch]
2022-09-16 16:34:29 -04:00
carlmagnus nordin
25d64f3d04 ZenLoader in editor:
* Fixed issue with LOAD_PackageForPIE getting lost when calling LoadPackageAsync from LoadPackageInternal
* Fixed issue with externally created packages not being tracked in the import store and unintentionally being loaded multiple times
* Fixed issue with instancing contexts not being propagated to imported packages

#preflight 63232218e872d7da0397f29e
#rb pj.kack
#jira UE-162812

[CL 22050271 by carlmagnus nordin in ue5-main branch]
2022-09-16 11:32:12 -04:00
joe pribele
73e907afca [CoreUObject] changed DuplicateDataReader/Writer to handle TObjectPtr
#rb zousar.shaker
#preflight 6320f8078c478acecf7abfaa

[CL 21998836 by joe pribele in ue5-main branch]
2022-09-13 22:44:01 -04:00
joe pribele
8dcd88cb17 [CoreUObject] changed the default behaviour of FArchiveUObjectFromStructuredArchiveImpl to forward FObjectPtr to the inner archive to avoid loading objects
Inner archive can choose how to handle FObjectPtr

#rb zousar.shaker
#preflight 6320ec888838676d1007de3e

[CL 21998399 by joe pribele in ue5-main branch]
2022-09-13 22:13:58 -04:00
Devin Doucette
0ce0756966 DerivedData: Added a way to request compressed data from derived data references
- FDerivedDataIoBatch::Compress requests the entire compressed value.
- FDerivedDataIoResponse::GetCompressedData gets the compressed buffer from the response when available.
- FDerivedDataIoResponse::GetCacheValueId has been added to get the FValueId when available.
- FDerivedDataIoResponse::GetCacheKey has been fixed to return a non-null cache key when appropriate.
- Asserts about FDerivedData now log the derived data reference too.

#preflight 631a5f6fa60c539c98ab5fa9
#rb Zousar.Shaker

[CL 21920523 by Devin Doucette in ue5-main branch]
2022-09-09 10:36:44 -04:00
carlmagnus nordin
da495608f8 ZenLoader: Fixed recently introdced atomic ordering problem
#rb pj.kack
#preflight 631b0aaeec45fbf3d7688182

[CL 21918456 by carlmagnus nordin in ue5-main branch]
2022-09-09 07:36:02 -04:00
paul chipchase
8d77e84166 Refactor the VA pushing API so that we only need to implement a single path
#rb Per.Larsson
#jira UE-163103
#rnx
#preflight 6318989c2b7fe03eb664e9f0

### VirtualizationSystem/VirtualizationManager
- Added an overload of ::Push taking just one FPUshRequest so that people don't have to keep adding MakeArrayView boiler plate when pushing a single request
- Change the order of the last two parameters for the raw ::Push call as this will group all of the payload specific parameters together and leave the target storage type at the end. It is unlikely that anyone is calling the older version but it has been deprecated for safety.

### IVirtualizationBackend
- The none FPushRequest overload of ::PushData is no longer virtual, it just converts the parameters to FPushRequest and calls that overload instead. In this way we now only have one pushing code path in our backends. We could probably look into removing this overload at this point (since the higher level IVirtualizationSystem will now convert all push requests into a FPushRequest form) but it is not considered worth it at the moment when the simple overload covers our needs.
- Removed EPushResult in favour of just returning true/false for the overall operation.If the caller needs a more detailed breakdown then they will have to use an overload that takes an FPushRequest over raw parameters.
-- At the moment FPushRequest does not contain a full breakdown of what happened, so with this submit we are effectively losing the ability to find out if the payload was already in the backends or not, however the batch version of push was already not returning this info so it is not a big loss. Fixing FPushRequest to return a better break down of what happened will be done in UE-160942
- Removed the none batch Push paths from the source control and ddc backends as they already supported batch pushing.
- File backend needed to be converted to supporting batch pushing, which is pretty much the same code as before except we need to iterate over the container of FPushRequests.
-- The backend does not early out on error as it tends to be quite fast. We might want to consider an official policy for the VA system, if we should early out of errors or not.

[CL 21907558 by paul chipchase in ue5-main branch]
2022-09-08 19:35:36 -04:00
carlmagnus nordin
b19634db1d ZenLoader: Removed incorrect check
[CL 21810130 by carlmagnus nordin in ue5-main branch]
2022-09-06 05:37:18 -04:00
carlmagnus nordin
5f42747bea Compile fix to match modified API
[CL 21790089 by carlmagnus nordin in ue5-main branch]
2022-09-05 04:36:35 -04:00
carlmagnus nordin
e582c065d7 ZenLoader: Initial support for loading packages in the editor
#rb pj.kack
#preflight 6310b0d1ea685939704bded6, 6311a3fa7778595cd69d7f6b
#jira UE-162812

[CL 21790081 by carlmagnus nordin in ue5-main branch]
2022-09-05 04:36:25 -04:00
Per Larsson
2f11775439 Bulk Data - fix bug when merging I/O batch read request(s)
- This change fixes an issue when batching bulk data I/O requests where the bulk data is layed out contiguous in the chunk/file. Internally
the max read size was previously capped to the size of the bulk data but when merging read requests we read past the first bulk data
instance, i.e. from first to last. This is safe as long as the range is stored contiguously in the same chunk/file.

#rb Paul.Chipchase
#jira UE-162910
#preflight 6314d66ba20b67673bca3713

[CL 21788475 by Per Larsson in ue5-main branch]
2022-09-05 02:56:20 -04:00
carlmagnus nordin
98ad51633e ZenLoader: When sync loading a package only execute nodes required for that package to finish loading (resubmit after fix)
#rb pj.kack
#preflight 63131ab3e11efde08fbe1510
#jira UE-162812

[CL 21788177 by carlmagnus nordin in ue5-main branch]
2022-09-05 02:34:01 -04:00
bob tellez
ab72d85eb2 [Backout] - CL21736945. Probable cause of crash in autotest
[FYI] CarlMagnus.Nordin
Original CL Desc
-----------------------------------------------------------------
ZenLoader: When sync loading a package only execute nodes required for that package to finish loading
#rb pj.kack
#preflight 630f9f4498103d4521d4adc3

[CL 21767699 by bob tellez in ue5-main branch]
2022-09-02 17:09:35 -04:00
carlmagnus nordin
6b11785d55 ZenLoader: When sync loading a package only execute nodes required for that package to finish loading
#rb pj.kack
#preflight 630f9f4498103d4521d4adc3

[CL 21737098 by carlmagnus nordin in ue5-main branch]
2022-09-01 03:59:17 -04:00
henrik karlsson
f0d634a556 [Runtime Core]
* AsyncLoading - Fixed out-of-bounds bug where QueuedPackages[0] was accessed when array was empty

Fixes a incremental editor test

#rb Francis.Hurteau
#preflight skipped

[CL 21716490 by henrik karlsson in ue5-main branch]
2022-08-30 23:04:47 -04:00
pj kack
2162f301c3 AsyncLoading2: Lower the new NotifyRegistrationEvent ensure (added in 21696962) to a log message
#rb carlmagnus.nordin
#rnx

[CL 21716434 by pj kack in ue5-main branch]
2022-08-30 23:03:25 -04:00
Per Larsson
9825bbd11d Bulk Data - fix missing include
#rb trivial
#jira none
#preflight 630e67a8e54ec9d5818d3155
#rnx

[CL 21707772 by Per Larsson in ue5-main branch]
2022-08-30 15:44:01 -04:00
Per Larsson
60dc823b46 Bulk Data - inline reloading ensures
#rb Paul.Chipchase
#jira UE-162155
#preflight 630dfb9ea416f6df25d0c82c

[CL 21699599 by Per Larsson in ue5-main branch]
2022-08-30 09:38:27 -04:00
paul chipchase
32ff3e56b4 Add a new save flag 'SAVE_RehydratePayloads' which when used will force the package being saved to be rehydrated, so that any virtualized payloads will be downloaded and stored in the resulting package file.
#rb Per.Larsson
#jira UE-161373
#rnx
#preflight 630cc42cf92416fb9262cd70

[CL 21697726 by paul chipchase in ue5-main branch]
2022-08-30 05:36:19 -04:00
Per Larsson
6181aa421f Bulk Data - refactored batch API to better support Nanite streaming
#rb Paul.Chipchase
#jira none
#preflight 630c97370345de4ccf7f202b
#rnx

[CL 21675673 by Per Larsson in ue5-main branch]
2022-08-29 08:53:10 -04:00
paul chipchase
4d8982c66f Fix the generation of duplicate bulkdata identifiers when creating new levels from templates containing bulkdata payloads.
#rb Zousar Shaker
#jira UE-161013
#rnx
#preflight 6308859d5a76ca27a8c08c2f

- When loading a map from a template we can load light and shadowmaps that contain bulkdata. When loading these they will retain the supposedly unique identifier used in the template. However when we save the level it will become a new package so we need those identifiers to be unique.
- Add a new option to FLinkerInstancingContext that signals that editor bulkdata objects being loaded should generate new identifiers

[CL 21587137 by paul chipchase in ue5-main branch]
2022-08-26 05:09:53 -04:00