#rb Per.Larsson
#rnx
#jira UE-148223
#preflight 6268efeb2f53f9169aa9b5c5
- Moving this to use the format that the other command lines do will make it easier to find.
- Moved the parsing code from ::ApplySettingsFromCmdline to ::ApplyDebugSettingsFromFromCmdline to indicate that it is a debug command and not expected for production use.
- Removed ::ApplySettingsFromCmdline as we no longer have non-debug commandlines.
- There is no console command version because we cannot change the graph mid process.
[CL 19936091 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#rnx
#jira UE-148223
#preflight 6267c4e2853fdb6fddb20be3
- Remove the Core.ContentVirtualizationDebugOptions section from the ini files entirely.
- Remove the ::ApplyDebugSettingsFromConfigFiles method as we no longer load any debug values from the config files.
[CL 19923780 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#jira UE-148223
#rnx
#preflight 62666fc90634d0904cca8756
- It was probably a bad idea to have debug settings like this applied from the config file as it would be fairly easy for people to accidently forget about them or submit them.
- The single threaded command can now be set via the commandline on start up using the '-VA-XXX' synctax that the system has started to adopt.
- Additionally it has been exposed as a console command so that it can be toggles on and off at runtime.
- Removed 'ForceSingleThreaded' from BaseEngine.ini and removed the parsing code for this from FVirtualizationManager::ApplyDebugSettingsFromConfigFiles.
- Removed 'FailPayloadPullOperations' from BaseEngine.ini, it hasn't been a valid setting for a while.
- Changed ConsoleCommands to ConsoleObjects and use it to store the base class IConsoleObject instead so it can contain console variables
- When pushing payloads we now only take the singlethreaded lock after checking if we have any backends/are enabled.
[CL 19900622 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#jira UE-148223
#rnx
#preflight 62629b8fbc24759dc73452dc
- Moved the debug values to their own structure to make it clearer that they are for debug purposes.
- Moved the code registering the console commands to a specific method.
[CL 19898824 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-148223
#rnx
#preflight 626252fed2644b9ff7d3167b
- Added verbose logging to show when a payload is skipped due to the miss chance, or when a backend skips a payload due to the 'VA.MissBackends' command
[CL 19862821 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-148223
#rnx
#preflight 6260f6e4360b45c32a84fae8
- The commandline -VA-MissBackends has been extended to a console command 'VA.MissBackends' that can be used at runtime to disable specific backends or reset the list and enable all of them again.
-- Entering VA.MissBackends without any args will print help documentation to the console/output log and show the user the names of the backends that they can disable.
- Technically we don't need the command line parsing anymore and users could use -dpcvars=VA.MissBackends=X instead but since we already have the code we might as well leave it untouched.
- Disabling backends from pulling payloads for debug purposes has been split off to it's own system stored as a
- IVirtualizationBackend::EOperations has been turned into bit flags, so we no longer need the 'both' entry
- IVirtualizationBackendnow has two EOperations members, one to record the operations that the backend supports and another that allows the operations to be disabled for debugging purposes.
- IVirtualizationBackend now has one method to poll if an operation is supported ::IsOperationSupported, which should not change over the lift time of the backend. In addition there is also a method for disabling/enabling a backend for debugging purposes as well as polling the debug state.
[CL 19845175 by paul chipchase in ue5-main branch]
#rb Per.Larsson, Sebastian.Nordgren
#jira UE-147948
#preflight 6256ba2f647ad886b380d007
- In some cases people are getting minor connection failures to the perforce server which causes the p4 print command to fail. In these cases it would be better to retry the command rather than failing to get the payload entirely
- Currently the default will be 2 attempts to pull a payload with a 100ms wait between them, this can be changed when setting up the backend graph as optional config values "RetryCount=" and "RetryWaitTime="
-- Note that RetryWaitTime is in ms
- Moved the parsing of the command line to it's own method in the backend.
[CL 19752733 by paul chipchase in ue5-main branch]
#rb trivial
#rnx
#preflight 623c5912c3399da9533ffd30
- The package trailer will be on by default soon and the option to remove it disabled, so this check is a bit pointless anyway.
[CL 19493442 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#rnx
#preflight 623ae40e7b69b01ec15da75c
#robomerge FNNC
- We need to expose the code that checks a set of packages for non virtualized payloads and virtualizes them so that the stand alone submission tool can access it.
- Currently I am adding this to the IVirtualizationSystem interface to avoid adding additional interface, but we might want to consider moving this some day as it is not a great fit with the rest of the interface.
[CL 19479757 by paul chipchase in ue5-main branch]
#rb Sebastian.Nordgren
#rnx
#preflight 622f645a19287beb2c359176
- Use the SetRequireDirPathEndWithSeperator option with the FUpdateStatus operation as we are checking depot file paths. If we do not use this option they can be seen as network paths and will be checked against the file system which can take some time to time out.
- Wrapped this up in a utility function so that it can be more easily updated in the future.
[CL 19373558 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#rnx
#jira UE-136480
#preflight 62289b59f4469cadac10eb91
- The backend can now connect to source control without forcing the rest of the editor to connect to source control.
- If the user disconnects the editor from source control, the backend will still be able to function using it's own connection.
-When the backend needs to change the current workspace in order to submit payloads we no longer invalidate the file cache used by the editor and so the user will no longer have to manually refresh files in the content browsers to restore the source control status icon.
[CL 19318970 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#rnx
#preflight 62276138671c913c0515c3b4
- To make it easier to extend the number of parameters when initializing the virtualization system, the function has been changed to accept a single struct, FInitParams, which will contains all potential parameters.
- Calling the version of UE::Virtualization::Initialize without any parameters will fallback to using the default values.
- The virtualization manager now passes the provided project name onto each backend that it creates.
- The source control backend now stored the provided project name and uses that when creating the submit description for new payloads rather than polling FApp for the current project name.
-- This is required for the stand alone virtualization application which will not have a specific project set.
[CL 19303638 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#rnx
#preflight 6225cf4cb57e715222ca15aa
- IVirtualizationSystem now has a new ::Initialize method which is called after it is first created. This allows the system to return a proper error when setting up which was not possible when the set up code ran in the constructor.
- The method also takes a FConfigFile reference which should be used when parsing settings from the config file system.
-- The config file being passed to this method, is either the optional config file provided when calling UE::Virtualization::Initialize, or (if that was not set) we find the GEngineIni config file from the global cache to use.
- This should ensure that the entire virtualization system is now loading the settings from the same set of config files where as before the code was a bit inconsistent (loading it in place, using the GConfig global, or trying to find the file from the global cache)
- If the system fails to create we now log if the problem was that we could not find a valid factory for the given system (meaning the module the system lives in was not yet loaded) or if the initialization phase failed.
- FVirtualizationManager now passes a single FConfigFile into all of it's methods that are parsing settings from it.
[CL 19283763 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-142926
#preflight 621e1f177f2803279b575764
- Now when payloads are submitted, we will submit them from within the project's saved directory (by default) this now works because we write a .p4ignore file to the submission directory which will override any additional ignore file higher up in the directory structure.
-- A new utility function ::CreateSubmissionSessionDirectory was added to make sure that the directory we are submitting from has a p4ignore file. It is added in it's own scope as it is assumed that additional functionality will be added here later.
- If a user wishes to choose a different location to submit from, they can override the environment variable "UE-VirtualizationWorkingDir" to provide a different location. This might be useful if the user does not have a huge amount of spare space on the disk that the project is running from.
- There is also an optional entry to the ini file set up 'SubmitFromTempDir' (false by default) which when set to true will cause the system to attempt to submit the payload files from the machines temp directory instead.
-- There isn't an easy way for users to override this locally, which reduces the usefulness of the option. This might be worth improving in the future although that time would be better spent removing the need for the directory in the first place.
- We now try to find/create the root directory to submit payloads from when the backend is created. Failure at this point is not expected and will prevent the backend from working and so is considered an error.
- Added more logging to track the session guid when submitting and to be clear about where the files are being submitted from.
[CL 19197367 by paul chipchase in ue5-main branch]
#rb Sebastian.Nordgren
#rnx
#jira UE-143680
#preflight 621776a4335015645d967662
#lockdown mark.lintott
- Strings inside of a LOCTEXT macro must be on a single line.
#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 19114310 in //UE5/Release-5.0/... via CL 19114561
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)
[CL 19152877 by paul chipchase in ue5-main branch]
#rb Sebastian.Nordgren
#jira UE-143686
#rnx
#preflight 62189ed68849a7424f7d2e03
- Workspaces created to submit virtualized assets are now prefixed as "VASubmission-" rather than "MirageSubmission-"
-- Use of the term mirage is deprecated and should be replaced by VA or VirtualizedAsset(s)
- Set the description when we create the workspace to info the user that it was auto generated during the submission of virtualized payloads.
[CL 19143761 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-142748
#rnx
#preflight 620e06d53e74a28b341e72b6
#lockdown aurel.cordonnier
- Turns out that the option did work fine, it just needed to be tested properly, so removed the assert that was guarding against mixed package trailers when submitting virtualized assets.
- Improved verbose logging when a payload is filtered based on payload size.
#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 19073541 in //UE5/Release-5.0/... via CL 19090494
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)
[CL 19134901 by paul chipchase in ue5-main branch]
#rb trivial
#jira UE-134434
#rnx
#preflight 620d1a10742ffef420223b15
- Changing a number of log items from 'Log' to 'Display' so that the info shows up more easily.
- Removed a number of log items that added no useful infomation
- Edited some of the existing log items to be clearer.
[CL 19031890 by paul chipchase in ue5-main branch]
#rb Sebastian.Nordgren
#jira UE-133472
#rnx
#preflight 620cef8f492761fc5cb53b05
- Because all of the work is blocking on the main thread adding text to ::EnterProgressFrame to better track progress can be misleading so for now we only display "Virtualizing Assets..."
[CL 19012877 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-133473
#preflight 620bf6ae483ff0ae5ec22257
### VirtualizationManager
- Added EPackageFilterMode that can be set via [Core.ContentVirtualization] in the config file to control the default virtualization behavior.
-- When set to EPackageFilterMode::OptOut(the default) all package paths will virtualize unless excluded by a pattern in UVirtualizationFilterSettings::ExcludePackagePaths.
-- When set to EPackageFilterMode::OptIn then no package path will virtualize unless included by a pattern in UVirtualizationFilterSettings::IncludePackagePaths.
- Added a TRACE_CPUPROFILER_EVENT_SCOPE to the constructor and ::MountBackends to better track the set up time costs.
- Change use of FConfigCacheIni::LoadLocalIniFile to use GConfig, there is no need to load our own.
- Improved verbose logging to show when a payload is rejected via filtering.
- We now early out if all payloads being requested in a push are rejected during validation.
- Renamed the FString overload for ::ShouldVirtualizePackage to ::ShouldVirtualize to make the difference clearer.
- Added support for UVirtualizationFilterSettings::ExcludePackagePaths when filtering
### UVirtualizationFilterSettings
- Now has a new FString array ExcludePackagePaths, which contains the paths/patterns used to force packages to be virtualized when filtering.
[CL 19010992 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#jira UE-141304
#rnx
#lockdown Mark.Lintott
#preflight 61fd3cccb5092d45ad18c953
- Not all backends gain from pushing payloads in batches. In this case the backend can opt not to override the default implementation which will just iterate over the batch of payloads and then push each one individually.
- However when the default path was used, FPushRequest::Status was not updated after the push was performed, which resulted in each request returning the default (failed) status.
-- The backends that use the default path are not enabled in any of our internal projects so this was not noticed.
- So even though the payloads were pushed, the virtualization process would log and error and block any submit being performed.
-- Subsequent submits of the same packages would work because we would note much earlier that the payloads are already stored in the backend and the broken code path would never run.
- Added fall through comment as required by the coding standards
#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18863318 in //UE5/Release-5.0/... via CL 18863334 via CL 18863548
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v910-18824042)
[CL 18863566 by paul chipchase in ue5-main branch]
