Commit Graph

5 Commits

Author SHA1 Message Date
paul chipchase
dc4779a5e3 Add a number of ways for the VA backend connections to be changed to lazy initialize on first use.
#rb Devin.Doucette
#jira UE-161599
#rnx
#preflight 6303c8d65a5d4e4624e7bf52

- There are some use cases that require the VA system to be initialized and configured correctly but would prefer that the backend connections only run if absolutely needed (usually when a payload is pulled or pushed for the first time), this change provides four different ways of doing this:
-- Setting [Core.VirtualizationModule]LazyInitConnections=true in the Engine ini file
-- Setting the define 'UE_VIRTUALIZATION_CONNECTION_LAZY_INIT' to 1 in a programs .target.cs
-- Running with the commandline option -VA-LazyInitConnections
-- Setting the cvar 'VA.LazyInitConnections' to 1 (only works if it is set before the VA system is initialized, changing it mid editor via the console does nothing)
--- Note that after the config file, each setting there only opts into lazy initializing the connections, setting the cvar to 0 for example will not prevent the cmdline from opting in etc.

- In the future we will allow the connection code to run async, so the latency can be hidden behind the editor loading, but for the current use case we are taking the minimal approach.
-- This means we only support the backend being in 3 states. No connection has been made yet, the connection is broken and the connection is working.
-- To keep things simple we only record if we have attempted to connect the backends or not. We don't check individual backends nor do we try to reconnect failed ones etc. This is all scheduled for a future work item.
- If the connections are not initialized when the VA system is, we wait until the first time someone calls one of the virtualization methods that will actually use a connection: Push/Pull/Query
-- We try connecting all of the backends at once, even if they won't be used in the call to keep things simple.
- Only the source control backend makes use of the connection system. The horde storage (http) backend could take advantage too, but it is currently unused and most likely going to just be deleted so there seemed little point updating it.
- If we try to run an operation on an unconnected backend we only log to verbose. This is to maintain existing behaviour where a failed backend would not be mounted at all. This logging will likely be revisited in a future work item.

[CL 21511855 by paul chipchase in ue5-main branch]
2022-08-23 13:01:15 -04:00
paul chipchase
52d01bd36f The virtualization system may now take the project name as a parameter when it is initialized rather than trying to find it itself.
#rb PJ.Kack
#rnx
#preflight 62276138671c913c0515c3b4

- To make it easier to extend the number of parameters when initializing the virtualization system, the function has been changed to accept a single struct, FInitParams, which will contains all potential parameters.
- Calling the version of UE::Virtualization::Initialize without any parameters will fallback to using the default values.
- The virtualization manager now passes the provided project name onto each backend that it creates.
- The source control backend now stored the provided project name and uses that when creating the submit description for new payloads rather than polling FApp for the current project name.
-- This is required for the stand alone virtualization application which will not have a specific project set.

[CL 19303638 by paul chipchase in ue5-main branch]
2022-03-08 11:22:45 -05:00
paul chipchase
b300426baf Add code comments to clearly state that UE::Virtualization::FHttpBackend has no long term future and that FDDCBackend should be used instead.
#rb trivial
#rnx
#preflight 61f91fc5a2514ba12f00b3ab

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18806979 in //UE5/Release-5.0/... via CL 18808619 via CL 18822203
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18822505 by paul chipchase in ue5-main branch]
2022-02-02 02:47:30 -05:00
paul chipchase
05d81ee7f3 EditorBulkData and the virtualization system now use FIoHash directly and FPayloadId is removed
#rb Per.Larsson, Devin.Doucette
#jira UE-133497
#rnx
#preflight 61f835491c5ac5523462810a

- A lot of files have been touched but most of this is changing FPayload::IsValid to !FIoHash::IsZero, the main changes are in EditorBulkData/EditorBulkDataTests
- The only difference between FPayloadId and FIoHash is that the former considered the hash of an invalid or empty buffer to be 'invalid' too where as FIoHash would return a valid hash
-- To keep behaviour the same, we only hash payloads in EditorBulkData using a utility method ::HashBuffer which will not hash empty or invalid payloads and return a default FIoHash instead.
-- Unit tests have been extended to prove that the behaviour has not changed.

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18806362 in //UE5/Release-5.0/... via CL 18808527 via CL 18821790
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18822154 by paul chipchase in ue5-main branch]
2022-02-02 02:21:24 -05:00
paul chipchase
a8d02fb07f Rename FJupiterBackend to FHttpBackend and reference sto Jupiter have been changed to Horde Storage
#rb trivial
#jira UE-136515
#rnx
#preflight 61b21e3b764790bee6bdb8b3

- This backend is still most likely going to be deleted before 5.0 ships but just in case it is not I am doing the renaming now to keep in step with the names we are using for Jupiter when communicating with externals.

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18419810 in //UE5/Release-5.0/... via CL 18422457 via CL 18422718
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v897-18405271)

[CL 18423123 by paul chipchase in ue5-main branch]
2021-12-09 14:58:09 -05:00