Add support for downloading a file from source control directly without actually checking it out (ie p4 print) and add additional support so that the command can be called from a background thread.

- Only implemented for perforce source control at the moment.
- The source control operation FDownload file can be used just like any other command via ::Execute BUT it will also work with the new method ::TryToDownloadFileFromBackgroundThread which unlike ::Execute can be called from a background thread.

#rb Sebastien.Lussier
#rnx
#preflight 60a2290322cce000014243ae

* PerforceConnection
- Extend ::RunCommand to take an optional FSharedBuffer reference which can be used to return text/binary data downloaded as part of a perforce command
-- Ideally we would replace all of the out parameters with a single struct but that level of refactor would be better suited to its own submit.
- Removed the bool parameter for FP4ClientUser and replaced it with a set of flags instead
- Added a flag to allow FP4ClientUser to store data, either in binary or text format from a perforce command. Although in general this will be the result of a p4 print command.

* PerforceSourceControlCommand
- No longer assert if the command is not on the game thread if the operation indicates that it is safe to run on a background thread.

* PerforceSourceControlModule
- Register the 'DownloadFile' worker

* PerforceSourceControlOperations
- Add FPerforceDownloadFileWorker

* PerforceSourceControlProvider
- Adding implementation for ::TryToDownloadFileFromBackgroundThread.
-- This method is thread safe compared to the normal ::Execute call due to:
-- 1) This method does not accept any callbacks as the most common use of callbacks with ::Execute is to update the UI.
-- 2) This method does not add the command to any of the internal queues nor rely on Tick to process the command.
-- 3) The worker is invoked directly from the calling thread rather than as a background task.
--4) This method does not allow commands that modify the cached states as this would be very difficult to manage without rewriting how all commands are processed to make sure that they are only run one at a time and the results processed in order. NOTE that async commands can be issued from the game thread and can in theory be run in any order on the p4 server depending on how the background task processing works out with no certainty as to the order the results will be processed in. It is unlikely that we will process results out of order but not impossible, but this is a pre-exisiting issue with the current API, extending it to be fully multithreaded will just make it worse.
-- Note that we do not explicity check if the command supports being called from a background thread, as creating the command will check that anyway.
- Calling ::OutputCommandMessages is now thread safe. When called from the game thread it will issue it's output via FMessageLog as before but when called from a background thread we will use the UE_LOG macros instead. Marshalling all output in the same way is something we will need to solve when/if we make the entire API thread safe.

* PerforceControlSettings
- Improve thread safety when calling ::GetConnectionInfo
- If the password is stored in the UI then this is not thread safe but that feature is hidden at the moment due to being unsecure. I am unsure if we want to move the password to be stored in a member variable that can be accessed from any thread and only updated via the UI or not.

* SourceControlOperations
- Add new FDownloadFile operation.
- This can either download the file(s) to FSharedBuffers which can be accessed by calling ::GetFileData OR directly to a directory on disk if the target directory is supplied as part of the constructor.

* ISourceControlOperation
- Add a new base method CanBeCalledFromBackgroundThreads (false by default) which can be overriden by derived classes and return true to indicate that the operation can be run from a background thread. This is intended for use by FDownloadFile with the perforce source control provider in lieu and making the API fully thread safe.

* PerforceSourceControl.uplugin
- Move the perforce plugin to load as early as possible as if Mirage is to use it then we need it to be avaliable before we start loading any potentially virtualized data in packages.

[CL 16346666 by paul chipchase in ue5-main branch]

Engine/Source/Developer/SourceControl/Private/ISourceControlProvider.cpp

@@ -98,4 +98,24 @@ TSharedPtr<class ISourceControlLabel> ISourceControlProvider::GetLabel(const FSt
 	return nullptr;
 }
 
+bool ISourceControlProvider::TryToDownloadFileFromBackgroundThread(const TSharedRef<class FDownloadFile>& InOperation, const FString& InFile)
+{
+	TArray<FString> FileArray;
+	FileArray.Add(InFile);
+
+	return TryToDownloadFileFromBackgroundThread(InOperation, FileArray);
+}
+
+bool ISourceControlProvider::TryToDownloadFileFromBackgroundThread(const TSharedRef<class FDownloadFile>& InOperation, const TArray<FString>& InFiles)
+{
+	// TryToDownloadFileFromBackgroundThread is unsupported by this source control provider
+	FFormatNamedArguments Arguments;
+	Arguments.Add(TEXT("ProviderName"), FText::FromName(GetName()));
+	FText Message = FText::Format(LOCTEXT("UnsupportedOperation", "TryToDownloadFileFromBackgroundThread is not supported by source control provider '{ProviderName}'"), Arguments);
+
+	InOperation->AddErrorMessge(Message);
+
+	return false;
+}
+
 #undef LOCTEXT_NAMESPACE

Engine/Source/Developer/SourceControl/Private/SourceControlOperations.cpp

@@ -0,0 +1,24 @@
+// Copyright Epic Games, Inc. All Rights Reserved.
+
+#include "SourceControlOperations.h"
+
+#include "Containers/StringView.h"
+#include "ISourceControlModule.h"
+#include "Misc/Paths.h"
+
+FDownloadFile::FDownloadFile(FStringView InTargetDirectory)
+	: TargetDirectory(InTargetDirectory)
+{
+	FPaths::NormalizeDirectoryName(TargetDirectory);
+
+	// Due to the asynchronous nature of the source control api, it might be some time before
+	// the TargetDirectory is actually used. So we do a validation pass on it now to try and 
+	// give errors close to the point that they were created. The caller is still free to try
+	// and use the FDownloadFile but with an invalid path it probably won't work.
+	FText Reason;
+	if (!FPaths::ValidatePath(TargetDirectory, &Reason))
+	{
+		UE_LOG(LogSourceControl, Error, TEXT("Path '%s' passed to FDownloadFile is invalid due to: %s"), 
+			*TargetDirectory, *Reason.ToString());
+	}
+}

Engine/Source/Developer/SourceControl/Public/ISourceControlOperation.h

@@ -69,6 +69,16 @@ public:
 		// Implemented in subclasses
 	}
 
+	/**
+	 * This will return true if the operation can be safely called from a background thread.
+	 * Currently it is assumed to only the operation 'FDownloadFile' will return true at least
+	 * until the API is made thread safe.
+	 */
+	virtual bool CanBeCalledFromBackgroundThreads() const
+	{
+		return false;
+	}
+
 	/** Factory method for easier operation creation */
 	template<typename Type, typename... TArgs>
 	static TSharedRef<Type, ESPMode::ThreadSafe> Create(TArgs&&... Args)

Engine/Source/Developer/SourceControl/Public/ISourceControlProvider.h

@@ -262,6 +262,32 @@ public:
 
 	virtual TArray<FSourceControlChangelistRef> GetChangelists( EStateCacheUsage::Type InStateCacheUsage ) = 0;
 
+	/**
+	 * Executes the FDownloadFile operation, but unlike the ::Execute method this can be called from a background thread, this
+	 * works because FDownloadFile is thread safe and it does not change the state of source control.
+	 * NOTE: This is only intended for use by the virtualization module and will be deprecated at some point in the future when
+	 * thread safety is built into the system.
+	 * NOTE: This is only implemented for the perforce source control provider with no plans to extend this to other providers.
+	 * 
+	 * @param InOperation The download operation to be executed.
+	 * @param InFile The depot path of a file to download.
+	 * @return True if the operation was a success, otherwise false.
+	 */
+	SOURCECONTROL_API bool TryToDownloadFileFromBackgroundThread(const TSharedRef<class FDownloadFile>& InOperation, const FString& InFile);
+
+	/**
+	 * Executes the FDownloadFile operation, but unlike the ::Execute method this can be called from a background thread, this
+	 * works because FDownloadFile is thread safe and it does not change the state of source control.
+	 * NOTE: This is only intended for use by the virtualization module and will be deprecated at some point in the future when
+	 * thread safety is built into the system.
+	 * NOTE: This is only implemented for the perforce source control provider with no plans to extend this to other providers.
+	 *
+	 * @param InOperation The download operation to be executed.
+	 * @param InFile An array of depot paths for the files to download.
+	 * @return True if the operation was a success, otherwise false.
+	 */
+	SOURCECONTROL_API virtual bool TryToDownloadFileFromBackgroundThread(const TSharedRef<class FDownloadFile>& InOperation, const TArray<FString>& InFiles);
+
 	/**
 	 * Whether the provider uses local read-only state to signal whether a file is editable.
 	 */

Engine/Source/Developer/SourceControl/Public/SourceControlOperations.h

@@ -3,8 +3,11 @@
 #pragma once
 
 #include "CoreMinimal.h"
-#include "SourceControlOperationBase.h"
+
+#include "Containers/StringFwd.h"
+#include "Memory/SharedBuffer.h"
 #include "ISourceControlChangelist.h"
+#include "SourceControlOperationBase.h"
 
 #define LOCTEXT_NAMESPACE "SourceControl"
 
@@ -630,4 +633,64 @@ public:
 	}
 };
 
+/**
+ * Operation used to download a file from the source control server
+ */
+class FDownloadFile : public FSourceControlOperationBase
+{
+public:
+	/** 
+	 * This constructor will download the files and keep them in memory, 
+	 * which can then be accessed by calling XXX.
+	 */
+	FDownloadFile() = default;
+	
+	/** This constructor will download the files to the given directory */
+	SOURCECONTROL_API FDownloadFile(FStringView InTargetDirectory);
+
+	// ISourceControlOperation interface
+	virtual FName GetName() const override
+	{
+		return "DownloadFile";
+	}
+
+	virtual FText GetInProgressString() const override
+	{
+		return LOCTEXT("SourceControl_PrintOperation", "Downloading file from server...");
+	}
+
+	virtual bool CanBeCalledFromBackgroundThreads() const
+	{
+		return true;
+	}
+
+	FString GetTargetDirectory() const
+	{
+		return TargetDirectory;
+	}
+
+	void AddFileData(const FString& Filename, FSharedBuffer FileData)
+	{
+		FileDataMap.Add(Filename, FileData);
+	}
+
+	FSharedBuffer GetFileData(const FStringView& Filename)
+	{
+		const uint32 Hash = GetTypeHash(Filename);
+		FSharedBuffer* Buffer = FileDataMap.FindByHash(Hash, Filename);
+		if (Buffer != nullptr)
+		{
+			return *Buffer;
+		}
+		else
+		{
+			return FSharedBuffer();
+		}
+	}
+private:
+
+	FString TargetDirectory;
+	TMap<FString, FSharedBuffer> FileDataMap;
+};
+
 #undef LOCTEXT_NAMESPACE