#rb Per.Larsson
#jira UE-189912
#rnx
- The filebased iostore system is tightly linked to the existing pakfile code and recieved notifications when a PakFile is mounted/unmounted directly in code. However we do not want to force the IAS system on all users, it should be opt in. So we need a generic way to hook into the PakFile system and be notified about PakFile operations.
- Add a new delegate (FCoreInternalDelegates::GetOnPakMountOperation) that is fired when ever a pakfile is mounted or unmounted.
- Add a new delegate (FCoreInternalDelegates::GetCurrentlyMountedPaksDelegate) that can be queried to find all of the pakfiles that are currently mounted.
- Both delegates are in a new internal header file to limit their use to the engine which will make it easier to change in the future if we need to do so.
[CL 30680178 by paul chipchase in ue5-main branch]
Exploits that the map is large (~1M) and rarely modified
Memory saved by:
* TMap saves slot and next-in-slot using 8B / item. FPackageIdMap is built once presorted, next-in-slot is next item. Only use 1bit / item to mark slot end.
* Only store 3/4 of FPackageId key - 1/4 is implicit by slot, which requires min 128K capacity
* Up to 400% load factor using 4-stage lookup, slot -> 2B hash -> 4B hash -> value. 2B array fits 32 items / 64B cache line, enabling a high load factor with little perf loss
* More compact entry handle and entry memory layout
* Deduplicating some entries
* Carefully storing entry data so container ownership is inferable from offset allows dropping 7.5MB of FIoContainerHeader::PackageIds and non-deduped StoreEntries
Surprisingly lookup got >4.5x faster too, despite 4 memory indirections instead of 2. Lookup can be optimized further by vectorizing. Rebuilding the map is also faster.
#rb pj.kack
[CL 29906389 by johan torp in ue5-main branch]
[REVIEW] [at]pj.kack
#rb pj.kack, john.huelin
#tests tested locally, also perf tested via replayrun exe override preflight in horde
#localization none
#preferred_allowlister john.huelin
[CL 29685404 by ben woodhouse in ue5-main branch]
FPakPlatformFile::ForeachPackageInIostoreWhile: Update usage of IterateDirectoryIndex, and use overload of TryConvertFilenameToLongPackageName using string views and builders to avoid allocations
#rb elias.lozadabenavent
#rb anton.dunchev
[CL 28282945 by nick edwards in ue5-main branch]
- Can be disabled with CVarAllowForceMonolithicShaderLibrary.
- FGameFeaturePreMountingContext can disable shader library loading in cases where we may want to control it externally.
- Install bundle manager now tracks if a bundle contains chunks
- GetPakchunkIndexFromPakFile now uses string views and string builders instead of temporary strings
- GetPakchunkIndexFromPakFile is no longer duplicated in IPlatformFilePak
- FPakPlatformFile::HandleMountPakDelegate no longer does a linear search to find the just mounted pak file
[REVIEW] [at]Dave.Belanger [at]Thomas.Ross [at]Bob.Tellez
[CL 28049947 by justin marcus in ue5-main branch]
Acquire list of pak files and then release pak-list lock.
Only release if we can acquire the reader lock without waiting.
#rb none
[CL 26864080 by robert millar in ue5-main branch]
* Moved dllexport from type to methods/staticvar in all Engine runtime code. This improves compile times, memory and performance in dll builds
[CL 26082269 by henrik karlsson in ue5-main branch]
Adding some Pak initialization and deinitialization logging to track down a horde only ensure.
#rb: trivial
[CL 25795014 by eric knapik in ue5-main branch]
I've reviewed all code using async requests and we had multiple places incorrectly handling the stat: some placed failed to decrease the counter, while others would decrease it when they shouldn't. The main cause was the fact that when we create an async request and pass nullptr as the target memory, the request allocates the memory itself and increases the STAT_AsyncFileMemory. However, when we call GetReadResults() on such requests, the responsibility is on the caller to balance the calls and decrease the stat, which is not very obvious and leads to the mentioned bugs. On top of that, when we call GetReadResults() we may not have the knowledge as to whether the returned memory was allocated by the request or not (which may affect whether we should touch the stat at all).
Resolved by adding IAsyncReadRequest::ReleaseMemoryOwnershipImpl member, which may be used by request implementations to do the proper clean up (balance the stat). This way requests take the full responsibility for their changes and users of the requests don't need to know their implementation details.
#preflight none
#rb Patrick.Laflamme
#jira UE-185064
[CL 25664733 by Wojciech Krywult in ue5-main branch]
