Commit Graph

2 Commits

Author SHA1 Message Date
paul chipchase
fc827406e2 Add a number of ways for the VA system to be changed to lazy initialize on first use.
#rb Per.Larsson
#jira UE-161296
#rnx
#preflight 62fe3ce73d3fb466b229bcc0

- There are some usecases that require the VA system to initialize the first time it is accessed (usually the first time we attempt to pull a virtualized payload) rather than be initialized in the program start up. This change provides three different methods to achieve this:
-- Setting the define 'UE_VIRTUALIZATION_SYSTEM_LAZY_INIT' to 1 in a programs .target.cs
-- Setting [Core.ContentVirtualization]LazyInit=true in the Engine ini file
-- Running with the commandline option -VA-LazyInit

- If we detect that the source control backend is being initialized on a background thread we do not try to run the FConnect operation. The backend will still work but this does reduce the potential error checking on initialization. This is done because the FConnect operation currently only works on the main thread and to change this would be a bigger work item than we can schedule at the moment.
- UE::Virtualization::Initialize functions now take a EInitializationFlags enum as a parameter. This enum allows the call to ignore all lazy init settings and force the initialization immediately. This is useful for programs like the Virtualization standalone tool which just needs to start the system when needed.
-- The call to ::Initialize in LaunchEngineLoop passes in None and does not ignore lazy initialization.
-- Calls to ::Initialize in the UnrealVirtualizationTool however all use EInitializationFlags::ForceInitialize and ignore lazy initialization settings.
- Fixed an odd bug in UE::Virtualization::Initialize where the error path (if the config file cannot be found) was using a different start up code path.
- Add asserts when assigning to GVirtualizationSystem to make sure that it is null. This is not 100% safe but should catch some potential threading issues, if any.
- Add an assert after lazy initialization (IVirtualizationSystem::Get) to make sure that GVirtualizationSystem was assigned a valid object.
- Improve how we check for legacy values in [Core.ContentVirtualization]. We now support multiple allowed values.
- Added a way to poll if a VA system has been initialize yet or not, this allows us to avoid initializing a VA system if one has not yet been created and we try to:
-- Dump VA profiling stats after cooking
-- Send VA stats to studio analytics
- Note that currently using lazy init loading will probably cause the VA statistics panel not to work, this will be fixed in future work where we will allow the panel to register for a callback when the system is initialized.

[CL 21467510 by paul chipchase in ue5-main branch]
2022-08-19 19:15:42 -04:00
paul chipchase
0fd3ca3da4 Add a rehydration command to the stand alone virtualization tool, making it easier to reverse the effects of asset virtualization. Unlike previous processes, this one does not require that we load the package and will just manipulate the data storaged in the package trailer.
#rb Sebastian.Nordgren
#rnx
#jira UE-156436
#preflight 62c287f9a3568e30664eb94f

### VA Standalone Tool
- We now plan to add much more functionality to the tool than just virtualizing and submitting changelists, so to make this easier I am moving the tool towards a design where it should be fairly easy to add new functionality.
- Added FCommand, which is a base class for adding new functionality, simple derive from FCommand and hook it up at the appropriate locations.
-- In the future it should be possible for new command types to automatically register themselves to be initiated from the command line. There should be no need to edit UnrealVirtualizationToolApp to add a new command but this will be done as an additional work item.
-- At the moment FCommand comes with a number of utility methods to call that cover some common source control commands.
-- The original functionality has not yet been moved to the command system and so the code is a little bit weird at the moment. Updating older code to the new system will be done as an additional work item.
- FProject/FPlugin have been moved to their own code files.

### Rehydrate Command
- The rehydrate command will take a number of packages, check them out of source control and then attempt to virtualize them.
- At the moment the chekout logic is fairly basic, we just check out every package supplied, we don't check if the package is virtualized or not yet. This can be improved in additional work items. Ideally by the end of command the only packages that we have checked out should also be rehydrated.
- At the moment the command can either take a path of a specific package, a path of a directory to find packages in, or a changelist containing packages that should be rehydrated.
- A cleint spec (workspace) can optionally be provided, but if not supplied we will attempt to find a client spec for which to check out the packages.
- Currently we will check out the packages to the default change list.

### Rehydrate process
- Added the rehydration process in it's own code files in the virtualization module. Like the virtualization process this is exposed in a public header file and no via the Core interface which means it is very specific to our module/implementation.
- The process expects that the caller will have checked out any required packages from source control. It will treat being unable to update a package file as an error.
- Added PackageUtils.h/.cpp and moved some of the generic code from the virtualization process code there so that it can be shared by the rehydration process.

### Misc
Moving away from the using things like FPackagePath as that requires that the correct mount points have been registered for a project and at the moment (with the flakiness of FConfig*) it seems that the best idea would be to prefer absolute file paths where possible.

[CL 20982284 by paul chipchase in ue5-main branch]
2022-07-07 06:54:33 -04:00