- Simplify partial loading logic which is now only applied during flush if required
- During partial load, only remove the exact RequestID being flushed so that other requestid can be flushed independently
- Add missing timeout handling for postloads
- Add missing support to run linkerload thread-safe postloads on async loading thread
- Add unittest to make sure timeout is respected for postloads
- Add unittest to make sure behavior of IsReadyForAsyncPostLoad is respected until a flush occurs
- Add unittest for syncloads coming from thread-safe postloads and validate partial loading behavior
- Make unittest run in editor only as they rely on saving package before reloading them
- Refactor unittest for better reuse
#rb Francis.Hurteau
[CL 28799225 by danny couture in ue5-main branch]
Should also speed up loading a buffer with a lot of NULL objects considerably (see PR #6875).
#jira UE-92396
#PR 6875
#rb robert.millar
#ushell-cherrypick of 28497640 by kristof.morva1
[CL 28502883 by KristofMorva in ue5-main branch]
- Add partial flush support from loading thread
- Fix the ConvertFromType allowing to load the softobjectptr during its conversion to hardref even when called from ALT and zenloader is active
- Add unittest to exercise softobjectptr to hardref conversion in ConvertFromType
- Move asserts preventing flush from loading thread directly into legacy loader
- Added a few veryverbose logs that were missing for linker load
- Fix partial loading being applied to sync calls coming from a postload, preventing the object to be fully postloaded before finishing the sync load
#rb Francis.Hurteau
[CL 28373045 by danny couture in ue5-main branch]
- Further improve the partial load logic by avoiding some operations where possible
- Also add some very verbose logging that can help identifying load ordering issues
#rnx
#rb Francis.Hurteau
[CL 28312996 by danny couture in ue5-main branch]
- Add a new partial loading mode where flushing the requestID will exit after serialization is done (after exportdone but before postload)
- Automatically opt-in partial loading when an explicit load is called recursively from another load (treated as an import)
- During recursive loading, attach postload of the recursive load to the caller's postload
- Add loading unit tests to validate the new behavior for zenloader
- Add a way to detect which loader is currently active to run unit tests on zenloader only
#rnx
#jira UE-190649
#rb Francis.Hurteau
[CL 28227792 by danny couture in ue5-main branch]
[FYI] danny.couture
Original CL Desc
-----------------------------------------------------------------
[ZenLoader] Fix recursive loading behavior in zenloader by following more closely how legacy loading works
- Add a new partial loading mode where flushing the requestID will exit after serialization is done (after exportdone but before postload)
- Automatically opt-in partial loading when an explicit load is called recursively from another load (treated as an import)
- During recursive loading, attach postload of the recursive load to the caller's postload
- Add loading unit tests to validate the new behavior for zenloader
- Add a way to detect which loader is currently active to run unit tests on zenloader only
#rnx
#jira UE-190649
#rb Francis.Hurteau
[CL 28214800 by bob tellez in ue5-main branch]
- Add a new partial loading mode where flushing the requestID will exit after serialization is done (after exportdone but before postload)
- Automatically opt-in partial loading when an explicit load is called recursively from another load (treated as an import)
- During recursive loading, attach postload of the recursive load to the caller's postload
- Add loading unit tests to validate the new behavior for zenloader
- Add a way to detect which loader is currently active to run unit tests on zenloader only
#rnx
#jira UE-190649
#rb Francis.Hurteau
[CL 28184263 by danny couture in ue5-main branch]
- Improves performance by waking up GT as soon as the async loading thread needs something processed on it
- Improves performance by only entering the sleeping part when the GT is actually idle instead of all the time
- Reduce power consumption and contention between GT/ALT by favoring longer sleep instead of spinning on platform not supporting sub milliseconds sleeps
#jira UE-90979
#rb PJ.Kack
[CL 27954710 by danny couture in ue5-main branch]
[Backout] - CL27745134
[FYI] stan.hormell
Original CL Desc
-----------------------------------------------------------------
[Backout] - CL27731288 - CIS / Build Errors
[FYI] Josh.Adams
Original CL Desc
-----------------------------------------------------------------
- Deprecated GetSectionPrivate and FindOrAddSection, and accessors in FConfigFile that could return a non-const FConfigSection (this is so we can track modifications to config values)
- Added AddToSection, RemoveKeyFromSection, etc to replace directly accessing a FConfigSection
- Fixed up Epic code for the deprecations (at least the majority, some projects that aren't built by Horde presubmit may have some that we will address going forward)
#jira UE-194955
#rb david.harvey and various others
[CL 27923017 by josh adams in ue5-main branch]
PackageStoreOptimizer: Expose a static version of FindScriptObject and functions for calculating PublicExportHash.
FNameMap: Add a foreach iterator to provide a list of all data in the struct.
ZenPackageHeader: Move from private to internal, add a constructor that returns an error string rather than fatally asserting.
#rb Per.Larsson
#rnx
[CL 27855318 by matt peters in ue5-main branch]
[FYI] Josh.Adams
Original CL Desc
-----------------------------------------------------------------
- Deprecated GetSectionPrivate and FindOrAddSection, and accessors in FConfigFile that could return a non-const FConfigSection (this is so we can track modifications to config values)
- Added AddToSection, RemoveKeyFromSection, etc to replace directly accessing a FConfigSection
- Fixed up Epic code for the deprecations (at least the majority, some projects that aren't built by Horde presubmit may have some that we will address going forward)
#jira UE-194955
#rb david.harvey and various others
[CL 27745141 by stan hormell in ue5-main branch]
- Added AddToSection, RemoveKeyFromSection, etc to replace directly accessing a FConfigSection
- Fixed up Epic code for the deprecations (at least the majority, some projects that aren't built by Horde presubmit may have some that we will address going forward)
#jira UE-194955
#rb david.harvey and various others
[CL 27731364 by josh adams in ue5-main branch]
#rb Paul.Chipchase
#rnx
## Background
The on demand I/O backend doesn't store complete/hole I/O chunks in the local file cache, only the parts that has been requested by the game. There are currently
two ways for querying the I/O backend(s) whether a specific chunk exists or not. The primary one used by the texture streaming system is DoesChunkExist(ChunkId). This
method is called via FBulkData::DoesExist(). In order to be able to fufill this query for partially cached I/O chunks, two additional methods has been
added to the I/O dispatcher API. Currently private, only accessible by FBulkData.
- DoesChunkExist(ChunkId, OffsetAndLength)
- GetSizeForChunk(ChunkId, OffsetAndLength)
The method GetSizeForChunk is only ever used by the IAsyncFileHandle wrapper for bulk data and is a tricky compromise. The file handle
represents the hole chunk/file where the bulk data instance resides, i.e. .ubulk, .uptnl etc. Calling get file size on this file handle should return
the size for the entire chunk/file. It is technically possible to read outside of the bulk data range and this would result in the read request
coming back as failed, since the resolve function in the I/O backend wouldn't be found. The file handle wrapper for bulk data is currently only used
by virtual textures and should be deprecated in the long run.
[CL 27337787 by per larsson in ue5-main branch]
