Change IPackageResourceManager::OpenAsyncReadPackage to return the Format along with the IAsyncReadFileHandle, so that FAsyncArchive can abort with an error if it is a text archive. This is necessary for cooked editor, which tries to use FAsyncArchive when available, but has to support text-format packages.
#rb Francis.Hurteau
#rnx
#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 18259898 in //UE5/Release-5.0/... via CL 18259993
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)
[CL 18260204 by matt peters in ue5-release-engine-test branch]
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971
[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035
[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485
[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
#rb Danny Couture
#rnx
### Problem
- The crash would occur when the texture being since contains a reference to another texture (via composite) which is also in need of syncing on the users machine.
- After the files on disk have been updated the current packages are reloaded but the original version of the texture will still be in memory when the composite texture notifies that it has been reloaded causing a texture compilation job with the older now invalid texture.
-- VirtualizedBulkData relied on out of date packages not trying to access their payloads on disk once the file has been changed, but with the above scenario it was possible. There could also be other paths causing this (even if the access of out of date package payloads at this point is a waste of time)
### Fix
- The virtualized bulkdata objects now reference themselves with the FLinkerLoader if the payload is on disk, just as the old bulkdata system did. When file changing operations are invoked, the FLinkerLoader will inform it's referenced bulkdata to load the payloads off disk and hold them in memory.
-- Note that virtualized bulkdata was trying to avoid this memory bloat by design, but if we cannot be certain that the bulkdata will not try to access the payload after the file changes (as was thought) then we must take the payload into memory.
- Added a log message to VBD if we detect that the loaded payload does not match the VBD members expectations. This log message can either be an error or a fatal error depending on the define 'VBD_CORRUPTED_PAYLOAD_IS_FATAL' (defaulted to on)
- The registration to FLinkerLoader can be easily disabled by setting the define 'VBD_ALLOW_LINKERLOADER_ATTACHMENT' to 0
### Outstanding Issues
- Cloned bulkdata objects (via assignment) do not currently register themselves to the FLinkerLoader (the old bulkdata system did not) if we were to add this then we need to make sure that the FLinkerLoader registration is thread safe. This should be done as a separate work item so that we can unblock the current workflows that this changelist addresses.
-- Note that if we did not update cloned bulkdata to work with this system then we could drop the registration and just iterate over the package's bulkdata and force them to load into memory when required. However that would leave the cloned virtual bulkdata in an uncertain state and is not desirable.
- When saving a package, the virtualized bulkdata objects that it contains will change their members so that future payload access can come from the newly saved files, where as the old bulkdata system would force the payload to remain in memory for the rest of the editor session. If we want to keep this then we will need to try and make sure that the newly updated bulkdata can be registered so that it receives future notifications about the file being changed.
#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 17648118 via CL 17648147 via CL 17648161 via CL 17648163 via CL 17648175
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v875-17642767)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 17648184 by paul chipchase in ue5-release-engine-test branch]
* Decouple container concept from IoDispatcher
* Decoiuple PackageStore implementation from AsyncLoading2
* Restore ucas unmount fix that got kist when merrging from UE4
* Fix packages being left in the PackageStiore even after unmounting contaiiners
#rnx
#rb pj.kack, per.larsson
#preflight 61520cc52afc2d0001146ce7
#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17641845 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v874-17637634)
[CL 17642353 by carlmagnus nordin in ue5-release-engine-test branch]
#rb CarlMagnus.Nordin
#rnx
#tests Ran overnight preflights on several platforms, opened/cooked/staged/ran the oldest version of InfiltratorDemo that can be downloaded (4.11)
### ObjectVersion
- Add a new version enum EUnrealEngineObjectUE5Version.
-- This version number starts at 1000 which leaves more than enough for for EUnrealEngineObjectUEVersion to be expanded
- Even though very few changes (if any at all) to EUnrealEngineObjectUE4Version are expected there is a static assert to make sure that EUnrealEngineObjectUEVersion::AUTOMATIC_VERSION never overtakes EUnrealEngineObjectUE5Version::INITIAL_VERSION.
- Add a struct FPackageFileVersion that wraps around the version numbers and is used to store them instead of raw int32 values which was done before. This should make it easier to add new version numbers in the future if we desire (although this will cause problems in places that serialize the struct directly)
### FPackageFileSummary
- Adding a new entry to CurrentLegacyFileVersion at value -8 which shows the UE5 version being added. This lets us make the changes without needing to submit anything to UE4 Main.
- When loading a package that does not have a UE5 version, it will remain at 0.
- Added ::IsFileVersionTooOld and ::IsFileVersionTooNew to replace hardcoded tests in the code base for version validity. This will make it easier to make changes in the future.
- A few months ago most of the accessors of the version number were deprecated in favour of a version that did not contain the Engine number (ie UE4Ver -> UEVer in Archive) but to work with these changes the renamed methods now will return or accept the version as FPackageFileVersion rather than int32. The old UE4 methods will remain deprecated and direct licensees to use the new methods.
### Archive
- Now stores the version as a FPackageFileVersion rather than int32
### LinkerLoad
- Reports the larger version number if we detect a higher version number than we support. Note that this could cause an issue if the UE4 version is ever raised but helps keep the code simple.
### AssetData
- Need to add a new version here to manage existing data that only has the UE4 version
### EditorDomain
- We do not need to version the format, we can just invalidate existing editor domain entries via EditorDomainVersion
### EditorServer
- When reporting that a package is too old we report the UE4 version as that is the only version that can be older than VER_UE4_OLDEST_LOADABLE_PACKAGE
- When reporting that a package is too new it can be either the UE4 or the UE5 version so we print them together "UE4Ver|UE5Ver"
### ContentCommandlets
- The min and max resave versions have been kept as a single value, you will not be able to resave against different UE4 and UE5 versions at the same time. It doesn't seem like a useful feature and would greatly increase the complexity of the code.
- We will also only report the file version as a single value.
### ManifestUObject
- This class was setting an older obsolete version on purpose to try and maintain compatibility with older clients so we need to provide a way to create an older UE4 only version that will leave the UE5 version as unset.
### NetworkPlatformFile
- I was unable to test the code path in FNetworkPlatformFile::ProcessServerCachedFilesResponse as I am unsure how to run the game in a mode that will actually use it.
- When reading an older "CookedVersion.txt" that was saved with a single version, the reads will fail and this will count as a version change in the code so that all of the existing files will be deleted. The existing code would not give the user a log message when this happens and given the very small time window where this might happen caused by this change I have opted to leave this alone and not add any additional logging.
- If we do detect a version mismatch we will still only log the version number as a single version.
### CookOnTheFlyServer
- We now add each version number to the IniVersionMap rather than merge the version and license version as a key/value pair. This allows us to a) use both the UE4 and UE5 version numbers b) we now log a warning that the version values don't match when it is changed, previously since it was a key value we would log a warning about an additional setting instead.
-- I also added "vs" to the log message when values are mismatched to make the space between the two values being printed clearer.
#ROBOMERGE-OWNER: paul.chipchase
#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 17549459 via CL 17550236 via CL 17550238 via CL 17550582
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 17550583 by paul chipchase in ue5-release-engine-test branch]
We plan to implement an IPackageWriter for workspace domain editor files to reduce duplication in SavePackage, and implementations for that purpose do not need to handle cook functionality.
IOStore-specific functionality is not needed by the cooker, and is pushed into IPackageStoreWriter.
#rb Per.Larsson, CarlMagnus.Nordin
#rnx
#ROBOMERGE-SOURCE: CL 17247188 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)
[CL 17247191 by matt peters in ue5-release-engine-test branch]
