#rb mark.lintott
#jira UE-165045
#rnx
- Add a new event (GetAnalyticsRecordEvent) to the UE::Virtualization namespace. Different telemetry systems can hook into this and respond when the virtualization system tries to send a new analytics event.
-- This even accepts an enum class of flags to provide better context for the event. At the moment the only valid flag is to request that the telemetry system flushes the event to make sure that it is sent before returning control to the event. In this way we can know that events are safely recorded before terminating the current process.
- To begin with we are only adding failure events and not recording any successes
-- Send an analytics even when the initial connection to perforce fails, the event should note if the user opted to retry their connection (after entering new details) or if they wanted to skip the check and load the editor anyway.
-- Send an analytics event when a payload fails to pull, the event should note if the user opted to retry to pull the payload or quit the process.
[CL 31087065 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-203381
-Some users have reported seeing their long cooks fail due to a machine having an unreliable connection or networkcard. In these cases the network outage can be swiftly fixed but if VA failed to pull a payload before then, the cook will terminate and need to be restarted which can cost a lot of time.
- It was requested that we add an optional way to have the system retry payload pulling when running in unattended mode but also to wait for X time (usually many minutes) before trying again. If it is likely that the connection will be restored within those few minutes then waiting will be much less costly than restarting a cook.
- That payloads can be pulled on many threads at the same time makes the logic a little tricky, so rather than counting how many payloads have failed vs the retry counter we count how many times we've logged a message to the user as this logging is protected by a critical section and acts as a way to "group" together failed pulls that occur around about the same time. We then reset this counter to 0 if we detect a successful pull.
- It is possible that a pull fails because the payload is missing, in which case this logic will probably cause the counter to reset frequently and the error to not become fatal for quite some time (possibly until the cook has almost finished) but it is quite unlikly to occur and due to this I have favored erring towards simple code rather than trying to track individual payload failures vs grouped failures vs successful pulls.
- Note: That when backends fail to pull payloads they generally log errors, which will eventually cause most of our processes to return non zero to indicate failure. VA should not log errors while we are inside of a retry loop and only print out errors when we detect a problem that we cannot solve to avoid this. This is being addressed as it's own work item.
[CL 30925886 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#rnx
- When generating content browser thumnails GIsRunningUnattendedScript will be set to true which prevents us from showing the VA payload pull failed dialog, so if a payload does fail during thumbnail generation the process will just terminate rather than asking the user if they would like to retry the operation.
- We no longer consider 'GIsRunningUnattendedScript' when checking if the process is interactive.
- If the dialog is being displayed from the GameThread we can force GIsRunningUnattendedScript to false safely allowing the dialog to be displayed. If we fail a payload pull on a background thread while the GameThread is generating a thumbnail we will not be able to safely change GIsRunningUnattendedScript (which does not have proper thread guarding) so we will just have to accept that the dialog will fail to show and the process will terminate.
[CL 29747059 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-180383
#rnx
- With the new error flow we block the process on a failed pull and then quit the process if the connection cannot be recovered, this means that we never reach the code that reports which payloads failed.
- Added similar reporting to FVirtualizationManager::OnPayloadPullError so we can get a log of which payloads failed to pull.
-- A pull failure should not really be dependant on the payload itself so this info isn't that important but it can be useful to eliminate potential issues by allowing us to confirm that the payload does in fact exist in persistent storage and the user was just unable to access it.
[CL 28833473 by paul chipchase in ue5-main branch]
#rb Martin Ridgers
#jira UE-191791
#rnx
- Add a new ::IsProcessInteractive function to UE::Virtualization::Utils namespace that will return true if we think the current process is being run in a scenario where a user will be present and that it is okay to show an interactive error dialog.
- Use this new check to make sure that we do not try to display the "connection error" dialog on start up (even though as a slate dialog this will usually fail to display in non-interactive scenarios anyway)
- Use the new check to avoid displaying the "payload pull failure" dialog and default to the process shutting down instead.
-- Added some code comments to explain why we force the process to exit rather than logging a fatal error.
[CL 28183133 by paul chipchase in ue5-main branch]
[Backout] - CL27745134
[FYI] stan.hormell
Original CL Desc
-----------------------------------------------------------------
[Backout] - CL27731288 - CIS / Build Errors
[FYI] Josh.Adams
Original CL Desc
-----------------------------------------------------------------
- Deprecated GetSectionPrivate and FindOrAddSection, and accessors in FConfigFile that could return a non-const FConfigSection (this is so we can track modifications to config values)
- Added AddToSection, RemoveKeyFromSection, etc to replace directly accessing a FConfigSection
- Fixed up Epic code for the deprecations (at least the majority, some projects that aren't built by Horde presubmit may have some that we will address going forward)
#jira UE-194955
#rb david.harvey and various others
[CL 27923017 by josh adams in ue5-main branch]
[FYI] Josh.Adams
Original CL Desc
-----------------------------------------------------------------
- Deprecated GetSectionPrivate and FindOrAddSection, and accessors in FConfigFile that could return a non-const FConfigSection (this is so we can track modifications to config values)
- Added AddToSection, RemoveKeyFromSection, etc to replace directly accessing a FConfigSection
- Fixed up Epic code for the deprecations (at least the majority, some projects that aren't built by Horde presubmit may have some that we will address going forward)
#jira UE-194955
#rb david.harvey and various others
[CL 27745141 by stan hormell in ue5-main branch]
- Added AddToSection, RemoveKeyFromSection, etc to replace directly accessing a FConfigSection
- Fixed up Epic code for the deprecations (at least the majority, some projects that aren't built by Horde presubmit may have some that we will address going forward)
#jira UE-194955
#rb david.harvey and various others
[CL 27731364 by josh adams in ue5-main branch]
#rb none
- If we want to use slate dialogs we cannot use LazyInitConnections, however slate dialogs only work on programs that a) support slate and b) are running in some interactive form. Commandline programs, or the editor runnning in unattended mode or running a commandlet cannot make use of the slate dialog.
- If this new flag 'DisableLazyInitIfInteractive' is set to true and 'LazyInitConnections' is set to true then the VA system will lazy init its connections unless we are running an interactive program that could make use of the slate dialogs, in which case 'LazyInitConnections' is disabled for safety.
- Ideally we'd merge these two values into a single enum but I am hoping that we can change how we connect in the near term in which case we will simple deprecate 'DisableLazyInitIfInteractive'. The use case is pretty niche at the moment so this will probably be easier for the next stage of work.
[CL 26230486 by paul chipchase in ue5-main branch]
#rb trivial
#rnx
- Any new option added to engine.ini:[Core.ContentVirtualization] needs to also be added to an allowed list in FVirtualizationManager so that we do not flag it as a deprecated option. This is because we used to store the manager config values in that section as well.
[CL 26120522 by paul chipchase in ue5-main branch]
#rb none
#rnx
#preflight 645c95abaa3c584c0b1c91f0
- A url can now be provided via 'ini:Engine:[Core.VirtualizationModule]:ConnectionHelpUrl=""' that will be added to various connection errors. If the error message is via FMessageLog then the user will be able to click the url to go straight to the link.
- This sort of help will be very company specific, this url will allow teams to build up their own FAQ for their users.
- Currently used by the source control backend if the initial connection fails but will be expanded to other areas.
- The url is currently parsed and owned by the virtualization manager but at the moment the connection error is issued by each backend. So right now there is a static accessor allowing backends to get the value to use. This would not work properly for 3rd party backends and will be changed in the near term.
[CL 25424000 by paul chipchase in ue5-main branch]
#rb trivial
#jira none
#rnx
#preflight 6458e2596c35ad81e60f2fa9
- I thought I had already done this but I see nothing in the p4 history, so it is likely that I forgot to actually submit the change.
- Fix some additional warnings found under clang (C5219)
[CL 25370075 by paul chipchase in ue5-main branch]
#rb none
#jira UE-181418
#rnx
#preflight 6447c5482804595a04d1e375
- Each pull operation now takes a FText that can be filled in with additional info about any failures that will cause the new error dialog to be displayed.
-- At the moment we only have info to provide from the source control backend if the connection itself failed.
-- In the future we might want to completely overhaul our error logging and display all messages to the user but our logging system is not really built for that level of redirection at the moment.
- Only pass on the error if it came from a persistent backend, we don't need to worry the user about cached backends which are allowed to fail.
[CL 25178752 by paul chipchase in ue5-main branch]
#rb trivial
#jira UE-182205
#rnx
#preflight 643d3249a35280ed4fca3097
- If all payloads were already in the backend we would still record the time taken to query but do not increment the number of hits nor the size of the payloads. Without a stat for "payloads skipped" which we do not currently have any interest in adding, recording the time taken to query where no transfer takes place just ends up being misleading and looks like a bug.
[CL 25066475 by paul chipchase in ue5-main branch]
#rb none
#jira UE-182205
#preflight 643d2901c947f6523a2f5845
- Most backends support some form of existence check which is run before a push to prevent uploading data that is already there. Sometimes we want to ignore that check (bugs in the backend etc) and need a way to force that to happen.
- The new config option [Core.VirtualizationModule]:ForceCachingOnPull (default false) will do this when set to true.
- The file and DDC backends will respect the flag but for now the source control backend will continue to check to avoid accidental pointless revisions being added.
[CL 25066347 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#jira UE-182205
#preflight 643932b2211b661dc4f594c6
- The enums don't contain any entries at the moment, the goal of this change is to reduce the friction that future API changes might have.
- The older versions of PullData/PushData have been deprecated but this won't flag compiler warnings if people have derived their own implementations of IVirtualizationBackend, so in addition to deprecating the methods have been set to 'final' and have checkNoEntry implementations which is the best we can do at the moment.
-- It is unlikely that people have actually done this as the documentation on adding new backend types will only go out in 5.3.
[CL 25039959 by paul chipchase in ue5-main branch]
#rb none
#jira UE-180383
#preflight 64183a83691c5ebc155d21a2
- The payload pull error dialog now suports an additional custom line in the dialog that can be set via Engine:[Core.VirtualizationModule]:AdditionalPullErrorMsg. The intent is to allow teams to provide additional help and info to their users.
- The default error text is now localized.
[CL 24714178 by paul chipchase in ue5-main branch]
#rb none
#jira UE-180383
#preflight 64147be21c44ff98b969b1aa
- The new flow will display an error dialog to the user informing them of the problem and ask if they want to retry or quit the process. This is quite a shift if how we have been handling failed payload pulls up until and this is the first rough implementation so for now it is off by default.
- We only show one dialog at a time and if a new error occurs while we are waiting for user input, that thread is set to wait on the existing dialogs outcome.
[CL 24689856 by paul chipchase in ue5-main branch]
#rb trivial
#jira none
#rnx
#preflight 6413305ef81cce1dd8daae8b
- We no longer error if the values cannot be parsed from the config file, our defaults in BaseEngine.ini are the same defaults that we have in code anyway.
- We now trust that FConfigFile will not write to the variable passed into it on parse failure so there is no need to parse to a local variable first, then apply that to the member if the parse was a success.
[CL 24671648 by paul chipchase in ue5-main branch]
#rb PJ.Kack
#jira UE-177013
#preflight 63ece283956709374a03eb8a
- The existing config file option 'EnableCacheAfterPull' is now deprecated. If we detect that it is still in the config file we will warn the user about it but continue to apply the value to maintain compatibility.
- Add a new config file option 'EnableCacheOnPull' which controls if a payload is pushed to cached storage after it has been pulled from persistent storage. (Default true)
- Add a new config file option 'EnableCacheOnPush' which controls if a payload is pushed to cached storage before it is pushed to persistent storage. (Default true)
- We now store the info about caching in a single enum bitfield ECachingPolicy.
- Added an up to date comment section about ini file set up for FVirtualizationManager which was previously missing.
[CL 24230636 by paul chipchase in ue5-main branch]
#rb Per.Larsson
#jira UE-177015
#preflight 63eb71ea956709374ab61d5e
- So far nobody has proposed any worthwhile use case for virtualizing engine content given the following downsides:
-- We cannot ever do so internally as we would be unable to distribute the virtualized packages.
-- Projects and the engine packages cannot always be accessed via the same client spec, leading to problems if we try.
-- Engine content is typically small and not suited to virtualization in the first place.
- If we remove the option then we can exclude the engine packages from our VA commandlets which will give less confusing output (at the moment if you try to virtualize a project containing a single package we will report that several thousand packages were found as we need to parse engine content too)
- Deprecated the ini file options, if we detect them we will warn the user that the option is still present and that engine content now will never virtualize.
[CL 24208899 by paul chipchase in ue5-main branch]
