- Detect when loader is hung and dump its state if it happens
- Add support for full flush from inside the last loaded package's callback if it ever happens
- Add unittest for that use case
#jira UE-207196, UE-207324
#rb kevin.macaulayvacher
[CL 33195963 by danny couture in ue5-main branch]
It does not actually require any package summary information since it only relies on TotalExportBundleCount which is known already in InitializeAsyncPackageFromPackageStore.
Fix nullptr crash in FAsyncLoadingThread2::FindOrInsertPackage for the rare special case path where the following dependency is added:
ImportedByPackage->GetExportBundleNode(ExportBundle_Process, DependentExportBundleIndex).DependsOn(&Package->GetExportBundleNode(ExportBundle_DeferredPostLoad, DependsOnExportBundleIndex));
#rb danny.couture, Per.Larsson
#tests Running Client, Server, Editor, CookedEditor
[CL 33194567 by pj kack in ue5-main branch]
[FYI] danny.couture
Original CL Desc
-----------------------------------------------------------------
[ZenLoader]
- Detect when loader is hung and dump its state if it happens
- Add support for full flush from inside the last loaded package's callback if it ever happens
- Add unittest for that use case
#jira UE-207196, UE-207324
#rb kevin.macaulayvacher
[CL 33176615 by bob tellez in ue5-main branch]
- Detect when loader is hung and dump its state if it happens
- Add support for full flush from inside the last loaded package's callback if it ever happens
- Add unittest for that use case
#jira UE-207196, UE-207324
#rb kevin.macaulayvacher
[CL 33172749 by danny couture in ue5-main branch]
Original CL Desc
-----------------------------------------------------------------
Improve error reporting when making an invalid bulkdata stream request.
#rb Per.Larsson
#rnx
- If a user tries to make a read request from FBulkDataBatchRequest that goes past the end of the bulkdata's payload we will eventually assert, but only at the point of reading which is often quite far away (context wise) from the code that initiated the request.
- Previously we were asserting if the user made a read request that exceeded the size of the bulkdata payload but this did not take into account reading from an offset.
- Fixed the assert to take offset into account so we now asset on invalid reads at the point that the read is first requested. In addition I have added more debug info to the assert making it easier to tell from the log file what was incorrect about the read request.
- This shouldn't raise any new issues as it's effectively just moving an assert that already exists earlier in execution.
[CL 33168986 by paul chipchase in ue5-main branch]
- Deprecate GIsEditorLoadingPackage global and use accessors instead so we can ensure thread-safety
- Make GIsEditorLoadingPackage compatible with ALT
- Prevent GIsEditorLoadingPackage from being misused
#jira UE-203917
#rb kevin.macaulayvacher
[CL 33165370 by danny couture in ue5-main branch]
This change resolves issues where codepaths would look for `RF_WasLoaded` to understand if a package had completed loading or not when in fact the package has not been loaded, but instead may only be partially loaded. Specifically, LinkerLoad::ResolveDeferredDependencies was changed to explicitly not load loaded or already loaded packages.
Any code guarding access to an partially loaded object will now be correct if guarding against objects missing RF_WasLoaded. However this change does means code looking for partially loaded objects will now need to look for RW_WillBeLoaded specifically. I imagine the amount of code that cares about partially loaded objects is quite low in practice compared to knowing if an object is fully loaded. Follow-up work aims to remove RF_WillBeLoaded completely in favour of another mechanism to check for in-flight work UE-212618
A test System.Engine.Loading.LoadBlueprintWithCircularDependency has been added that exhibits a hang if RF_WillBeLoaded is not checked for in ResolveDeferredDependencies before calling LoadPackage since will cause a flush to happen potentially as part of an already on-going flush which can lead to partial loads being unable to be unblocked and the loader softlocking
#jira UE-211383
#rb Francis.Hurteau
#tests System.Engine.Loading.LoadBlueprintWithCircularDependency
[FYI] Francis.Hurteau
[CL 33141808 by kevin macaulayvacher in ue5-main branch]
#rb Per.Larsson
#rnx
- If a user tries to make a read request from FBulkDataBatchRequest that goes past the end of the bulkdata's payload we will eventually assert, but only at the point of reading which is often quite far away (context wise) from the code that initiated the request.
- Previously we were asserting if the user made a read request that exceeded the size of the bulkdata payload but this did not take into account reading from an offset.
- Fixed the assert to take offset into account so we now asset on invalid reads at the point that the read is first requested. In addition I have added more debug info to the assert making it easier to tell from the log file what was incorrect about the read request.
- This shouldn't raise any new issues as it's effectively just moving an assert that already exists earlier in execution.
[CL 33134247 by paul chipchase in ue5-main branch]
resubmit of earlier change 32762180 after rollback
modified to work around bug in FCompressedBuffer corrupting serialization when payload size is zero
#rb paul.chipchase
[CL 33007777 by charles bloom in ue5-main branch]
[FYI] charles.bloom
Original CL Desc
-----------------------------------------------------------------
EditorBulkData : serialize compressed data to the undo buffer without decompressing
#rb paul.chipchase
[CL 32890531 by charles bloom in ue5-main branch]
#rb Per.Larsson
#rnx
- If archive is corrupt we could serialize an unreasonable value for the payload size from disk. This could cause very large allocations to be made which could cause the process to OOM or to allocate resources that it doesn't need.
- To avoid this we can check the archive we are serializing from and make sure that, at the very least, it will be possible to load all of the payload bytes from it. This way we will never make an allocation larger than the remaining space in the archive. It would then be up to the higher level code to validate that overall size of the provided file makes sense.
- Note that we are not able to do this is we are loading a payload from legacy bulkdata format that was marked as compressed as we only know the uncompressed payload size, not the compressed size on disk.
[CL 32755840 by paul chipchase in ue5-main branch]
#rnx
[FYI] Steve.Robb
Original CL Desc
-----------------------------------------------------------------
Added FUtf8StrProperty and FAnsiStrProperty.
#rb robert.manuszewski
#jira UE-204743
[CL 32590075 by alex kahn in ue5-main branch]
The goal here is that if a UObject is constructed during a transaction, and then decides to abort, we leave the UObject in an intact-enough state that the GC can come along later to destruct and deallocate the object.
To that end, we move StaticConstructObject_Internal back to instrumented, but execute the UObject malloc and UObjectBase constructor uninstrumented. Then later when the derived constructor runs over the same memory, we run the constructor instrumented, except when it gets down to the UObject constructor level, we run that uninstrumented.
If we abort, the UObject memory will not be freed by the AutoRTFM runtime, the destructor will not be called, but the memory will be rolled back to what the UObject and base constructors initialize it as, so it should be a bare bones object ready for destruction and deallocation.
- added a new attribute to the compiler (autortm_always_open) so we can annotate functions themselves (like constructors) to ensure the entire function isn't instrumented (including code we can't normally wrap, like when the constructor writes the vtable pointer)
#rb Brandon.Schaefer, neil.henning
[CL 32426570 by michael nicolella in ue5-main branch]
- Move PreGarbageCollect broadcast outside of the GC lock because it's too easy for user code to cause deadlocks by calling into other part of the engine.
- Introduce a new event when GC has started and the GC lock is held
- Use the new event for zenloader as it provides thread-safety guarantees with async loading thread
#rb Robert.Manuszewski
[CL 32362280 by danny couture in ue5-main branch]
