#rb Per.Larsson, Matt.Peters
#jira UE-153904 , UE-156201
#preflight 62ce802c471a2c2886f592e8
### Problem
- Because of the package trailer we can easily find out which payloads are present in a package file and where they are. This means we don't need to rely on the cached offset into the file that bulkdata generally uses for payload loading. This means as long as the payload is still actually in a payload file then we can load it no matter what.
- We will still give errors in the following scenarios
-- The payload is not stored in a package trailer (either referencing older bulkdata or an older editor bulkdata format)
-- We cannot get a read handle to the package file.
-- The payload is no longer in the package file.
- If the attachment is still valid then we can guarantee that the payload can be loaded. Without an attachment we make our best efforts but in the case of the package file being reomoved/replaced this can fail. So ::ResetLoader will still be used to indicate that we want to remove the attachment but expect the package file to be changed and to load the payload into memory for safety. Work on improving this is another work item for the future.
### Fix
- We no longer remove the package path when ::DetachFromDisk is called nor do we remove flags relating to the payload. From this point onwards we use the attached archive being null to determine how we load. We continue to set OffsetInFile to an invalid value as we can no longer safely use it.
- Add a new method ::IsAttachedToPackageFile to explicitly show when we are checking to see if the bulkdata is still 'attached' to the package file.
- Explicitly set AttachedAr to null ptr when ::DetachBulkData is called on it. In theory the ::DetachBulkData call will call FEditorBulkData::DetachFromDisk which in turn will set AttachedAr but it can look confusing when reading the code. Explicitly setting the AttachedAr member to nullptr helps make the intent clearer.
- When loading the payload from disk there is now a new code branch that is followed if we detect that we are loading from a package file that we are no longer attached to.
-- This path gives bespoke errors in cases where it is not safe for us to try loading the payload, in addition there is a bespoke error message if loading from the package trailer fails.
-- Note we can only risk loading from the package trailer as it's format allows for us to find if the payload is still on disk or not. With the older formats we will not know if we are loading garbage or not.
-- Fixed a bug with CVarShouldLoadFromTrailer. When it is true we will only attempt to load payloads from the trailer if the payload is stored in the trailer. The comment for this development CVar has been updated.
- There has been a pass made to error reporting. If we are reporting an error relating to the package we will output the package name, if we are reporting on a error relating to the file itself then we will output the package path + extension
[CL 21089705 by paul chipchase in ue5-main branch]
The class part of an ExportText path now also starts with a /, so the code that was trimming it off during SetPath was no longer running.
#jira
#preflight 62cf05bb7822772758de6914
#rb Matt.Peters, Robert.Manuszewski
#rnx
#ROBOMERGE-AUTHOR: jamie.dale
#ROBOMERGE-SOURCE: CL 21078121 via CL 21080081 via CL 21080570 via CL 21080909
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)
[CL 21083083 by jamie dale in ue5-main branch]
#rb pj.kack
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 21060094 via CL 21060128 via CL 21060415 via CL 21060535
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)
[CL 21078246 by robert millar in ue5-main branch]
This covers the following types and meta-data:
* Blueprint Generated Class (class, property, and function meta-data)
* User Defined Struct (struct and property meta-data)
* User Defined Enum (enum meta-data)
#preflight 62cc7d31173a4f3408716829
#rb Francis.Hurteau
#ROBOMERGE-AUTHOR: jamie.dale
#ROBOMERGE-SOURCE: CL 21057147 via CL 21057174 via CL 21057182 via CL 21057190
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)
[CL 21078081 by jamie dale in ue5-main branch]
The fix was causing other side effects included non deserialized objects in some cases
#rb PJ.Kack
#ROBOMERGE-OWNER: francis.hurteau
#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 21055748 via CL 21056031 via CL 21056040 via CL 21056045
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)
[CL 21076940 by francis hurteau in ue5-main branch]
Do not validate optional reference to cooked data when the cooked context is empty
#rb Jamie.Dale, Matt.Peters
#preflight 62cc8979604402cc47eea7ba
#ROBOMERGE-OWNER: francis.hurteau
#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 21055685 via CL 21055920 via CL 21055940 via CL 21055961
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)
[CL 21076921 by francis hurteau in ue5-main branch]
#rb PJ.Kack
#rnx
#preflight 62cc18391961b487b3757b5e
- MSVC Static Analysis picked up that the Class pointer was still being used in the branch where we know Class == nullptr.
- When iterating over the pending CDOs and logging them, we should be logging the class name for that CDO.
- Added an assert to check that CurrentClass is valid before using it.
- Fixed some code alignment issues that occurred in a recent code merge.
#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 21053641 via CL 21053644 via CL 21053647
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)
[CL 21075787 by paul chipchase in ue5-main branch]
Added a FCoreDelegates delegate called by the BP reinstancing path. The existing one is called per-batch of reinstanced objects, but this means that some objects will still be REINST_ at the point of the call. This was causing ordering issues when attempting to re-link linked anim graphs (which point to object internals). The new delegate is called once all references have been replaced so the reinstanced objects should be self-consistent.
Reworked reinstancing logic to now account for this self-consistency. Key to this is the caching of the 'source' object (i.e. the host anim instance) on nodes that inherit from FAnimNode_CustomProperty - this allows the reinstancing logic to correctly handle linking & property patchup at the point of the delegate call.
Fixed a memory stomp when compiling an anim BP that contains a control rig node. This was caused by the control rig's delegate still being pointed at the old instance due to the lambda binding. Switched this to a raw binding that is cleaned up on node destruction.
#jira UE-158849
#rb Phillip.Kavan,Jurre.deBaare
#preflight 62cec541d00f3c208e446dce
[CL 21070551 by Thomas Sarkanen in ue5-main branch]
Initial submit, compiled out by default.
#rb Peter.Engstrom,Mattias.Hornlund,Brian.Bekich,Ryan.Gerleve,John.Barrett
#jira UE-158381
#preflight /62cd6fc7ad490b52da0515a3
[CL 21068370 by Peter Engstrom in ue5-main branch]
#rb Sebastian.Nordgren
#rnx
#jira UE-156436
#preflight 62c287f9a3568e30664eb94f
### VA Standalone Tool
- We now plan to add much more functionality to the tool than just virtualizing and submitting changelists, so to make this easier I am moving the tool towards a design where it should be fairly easy to add new functionality.
- Added FCommand, which is a base class for adding new functionality, simple derive from FCommand and hook it up at the appropriate locations.
-- In the future it should be possible for new command types to automatically register themselves to be initiated from the command line. There should be no need to edit UnrealVirtualizationToolApp to add a new command but this will be done as an additional work item.
-- At the moment FCommand comes with a number of utility methods to call that cover some common source control commands.
-- The original functionality has not yet been moved to the command system and so the code is a little bit weird at the moment. Updating older code to the new system will be done as an additional work item.
- FProject/FPlugin have been moved to their own code files.
### Rehydrate Command
- The rehydrate command will take a number of packages, check them out of source control and then attempt to virtualize them.
- At the moment the chekout logic is fairly basic, we just check out every package supplied, we don't check if the package is virtualized or not yet. This can be improved in additional work items. Ideally by the end of command the only packages that we have checked out should also be rehydrated.
- At the moment the command can either take a path of a specific package, a path of a directory to find packages in, or a changelist containing packages that should be rehydrated.
- A cleint spec (workspace) can optionally be provided, but if not supplied we will attempt to find a client spec for which to check out the packages.
- Currently we will check out the packages to the default change list.
### Rehydrate process
- Added the rehydration process in it's own code files in the virtualization module. Like the virtualization process this is exposed in a public header file and no via the Core interface which means it is very specific to our module/implementation.
- The process expects that the caller will have checked out any required packages from source control. It will treat being unable to update a package file as an error.
- Added PackageUtils.h/.cpp and moved some of the generic code from the virtualization process code there so that it can be shared by the rehydration process.
### Misc
Moving away from the using things like FPackagePath as that requires that the correct mount points have been registered for a project and at the moment (with the flakiness of FConfig*) it seems that the best idea would be to prefer absolute file paths where possible.
[CL 20982284 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#rnx
#preflight 62c6a9faf85287873280cef7
EFlags::TransientFlags has been removed from the enum and instead is stored as a constant. This will allow the debugger to more easily display the flag member.
Move the removal of transient bulkdata flags when saving to ::BuildFlagsForSerialization so that all flag manipulation is done in the same place.
We now update the flags at the end of serialization and remove the EFlags::IsVirtualized bit (if set) if the payload is being stored in a package trailer
-- I originally tried to do this by removing the bit in ::BuildFlagsForSerialization, but that was partially reverted in CL 19976045 as it was causing the editor bulkdata is rehydrate as we check for EFlags::IsVirtualized in several places when saving out the bulkdata
-- Moving the bit removal to the end of serialization is the easiest and safest way to fix this. However with some more work we could probably remove more branches from this logic but I'd rather move that to a separate work item.
Moved the paranoid checks to make sure that the package trailer builder was created correctly to a static method ::ValidatePackageTrailerBuilder in order to make the ::Serialize method easier to read.
[CL 20982025 by paul chipchase in ue5-main branch]
