Always silence warning about missing package for script packages, just like GetPackageLinker() simply returns nullptr for PKG_InMemoryOnly.
submitted on behalf of [at]PJ.Kack
#rb Matt.Peters
#preflight 63fe8765ae54ee4ce9e59742
[CL 24469377 by francis hurteau in ue5-main branch]
#rb Per.Larsson
#preflight 63f8922ab879197e73369f7e
- CL 24316937 made an effort to prevent serialization of bad data from corrupting memory, however the checks were a little too zealous. In some cases the archive provided for serialization is already in an error state in which case we should trust that the calling code has processes the problem correctly. Handling the error at this point from within bulkdata can give misleading errors.
- We now check if the archive is in the error state or not BEFORE we start serialization and if it is in error we just return early with default values.
- This should ensure that the problem is reported by the system creating it rather than always by bulkdata and make things easier to diagnose/fix/
[CL 24406791 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#rnx
#preflight 63f3b1fa52460c3a1f4684ce
- The problem is that our allocation functions take allocation sizes as type SIZE_T which on older 32bit platforms will be int32, but bulkdata stores sizes as int64. If the requested allocation is large enough the size will be truncated and the allocation could still return a valid memory allocation but one much smaller than requested. Later when we try to load into that allocation we will overwrite the end of the buffer. Given our supported platforms and how bulkdata is generally used this is an unlikely scenario but one we still should protect against.
- Added two functions ::SafeMalloc and ::SafeRealloc to replace direct calls to malloc/realloc. These utility functions will check to make sure that we can make allocations of the requested size by making sure that SIZE_T (the type that our allocators take for allocation sizes) is a large enough type.
- If the platform cannot service the request we will issue a fatal log error and halt the process. Technically we could error and return nullptr but in practice very little code checks to see if bulkdata allocations are valid or not and will end up crashing anyway. Given that this problem is only going to show up on 32bit machines and only for very large assets (most likely invalid or broken ones at that) we might as well halt execution as close to the problem as possible to make debugging easier.
- The changes are only applied for platforms where PLATFORM_32BITS is true, should be no cost to 64 bit platforms.
[CL 24335588 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#jira UE-172853
#rnx
#preflight 63f3834a61378b7ea0065346
- The main issue we are trying to solve is that we serialize the number of elements when saving/loading the package. On load we multiple the number of elements (int64) by the size of each element (int64) and if either of those values is large enough we could overflow and end up with an incorrect size. We do check for this when saving the package so this is only likely to come up if the package has been corrupted in some way.
- Change FBulkMetaData::FromSerialized to do the actual serialization as well. This lets us set the error flag on the archive if we do detect problems and keep all the error handling in a single place.
-- Note that setting the error flag on an archive while a package is being loaded doesn't seem to do much at the moment, so this is more of an attempt to future proof.
- If we detect an error with serializing FBulkMetaResource we will log an error and immediately return so that FBulkMetaData will remain set to defaults.
- We now calculate the largest value of ElementCount that we can support, given the ElementCount size and if the count is exceeded we will log an error, set the error flag on the archive and return leaving FBulkMetaData as default.
- Note that both before and after this change the package is effectively broken and won't work properly, but this change will stop us from potentially stomping memory.
- Note that at runtime we generally serialize the bulkdata info from the BulkDataMap in the package header, so this is mostly an editor thing.
[CL 24322887 by paul chipchase in ue5-main branch]
added because FArchive by default asserts on FObjectPtr
#rb zousar.shaker
#preflight 63ebc8e4b64ac0eccbfda48a
[CL 24215955 by joe pribele in ue5-main branch]
- Fixes cooked softobjectpath fixup on instanced WorldPartitions when not using iostore
#rb jamie.dale, francis.hurteau
#preflight 63d813185c69f453c11796a1
[CL 23927477 by patrick enfedaque in ue5-main branch]
#rb carlmagnus.nordin
#rnx
#preflight 63ca7b97e3138815655cf971
- Changed a ranged for loop in FEventLoadGraph::RemoveNode to be index based to avoid pointer arithmetic to index casting.
- FAsyncPackage::PostLoadDeferredObjects and FAsyncPackage::CreateClusters were not modifying the parameter InOutTimeLimit that was being passed in by reference, so it has been changed to be passed by value to avoid confusion.
- Changed a number of methods that were taking float values to doubles, in most cases double variables were being passed into these methods and they were being compared to doubles inside of the methods, so there didn't seem to be much point in the parameter type being float.
### Double->Float explicit cast
- Need to do this when using the IAsyncReadRequest api.
- Need to do this when using FLinkerLoad
- FAsyncLoadingThread::TickAsyncThread otherwise we'd need to change FAsyncLoadingThread::ProcessAsyncLoading to take doubles, which in turn would require a large number of double->float casting when calling out to the rest of the engine.
[CL 23788745 by paul chipchase in ue5-main branch]
- extracted out callstack offets and stack traces into FArchiveCallstacks
- created FArchiveStackTraceWriter that wraps an archive and writes into a FArchiveCallstacks
- refactored existing FArchiveStackTrace to use the new writer and callstack collection
- added IPackageWriter::CreateLinkerExportArchive in order to create a stack trace archive writer in DiffPackageWriter
#rb Matt.Peters, Francis.Hurteau
#jira UE-173560
#preflight 63c7bd232e714f64ad89fd2c
#rnx
[CL 23787282 by Per Larsson in ue5-main branch]
#rb Matt.Peters
#jira UE-174329
#rnx
#preflight 63c95073ac35a0e9da7f2c30
- This replaces the solution to the same problem that was submitted in CL 21587137.
- Instead of trying to pass the need to regenerate the guids via FLinkerInstancingContext it is now done via ELoadFlags.
- The fix was originally added to FLinkerInstancingContext as a) at the time a similar fix had been made to instancing b) we felt it would be easier to remove in the future when we change how the guids are calculated making the need to regenerate moot. However this stopped working when 22635658 prevented the instancing context from being passed to objects not being instanced. The concept was a better fix for the flags so it has been moved there.
[CL 23787156 by paul chipchase in ue5-main branch]
