Files
UnrealEngineUWP/Engine/Source/Runtime/NetworkFile/Public/NetworkPlatformFile.h
Robert Manuszewski 301e5763c2 Copying //UE4/Dev-Core to //UE4/Dev-Main (Source: //UE4/Dev-Core @ 3228803)
#lockdown Nick.Penwarden
#rb none

==========================
MAJOR FEATURES + CHANGES
==========================

Change 3209807 on 2016/11/24 by Chris.Wood

	CRP now has improved handling of lost comms with website/DB (CRP v.1.2.9)
	[UE-38397] - CrashReportProcess should not lose crashes when the website goes down.

Change 3209936 on 2016/11/24 by Steven.Hutton

	Reconciled offline work. Fixed a number of display bugs with the reports page.

Change 3209938 on 2016/11/24 by Steven.Hutton

	Reconciled offline work

	Adding JQuery UI packages

Change 3210736 on 2016/11/28 by Steve.Robb

	Unset() made protected, which is an implementation details of TFunction and not the way to unbind one (may leak memory).
	Fixes to existing calls.

	https://answers.unrealengine.com/questions/494947/proper-way-to-deal-with-destroying-a-tfunction.html

Change 3211181 on 2016/11/28 by Steve.Robb

	Improved error message when binding a delegate to a pending kill object.

	#jira UE-5232

Change 3211214 on 2016/11/28 by Steve.Robb

	PR #2978: fixed typo in FMallocLeakDetection (Contributed by finap)

	#jira UE-39049

Change 3211301 on 2016/11/28 by Steve.Robb

	PR #2892: Copy bCustomVersionsAreReset when creating an Archive from another Ar. (Contributed by surakin)

	#jira UE-37941

Change 3213387 on 2016/11/29 by Steven.Hutton

	Test of a release note parsed from the changelist

Change 3213553 on 2016/11/29 by Gil.Gribb

	UE4 - Rework dependencies and UStruct creation so that we can create UBlueprintGeneratedClass's without needing the parent class serialized.

Change 3214800 on 2016/11/30 by Robert.Manuszewski

	ModuleManager will now use a critical section instead of FMultiReaderSingleWriterGT to make it really thread safe.

	- removed FMultiReaderSingleWriterGT because it was not fully thread safe

Change 3214926 on 2016/11/30 by Robert.Manuszewski

	Merging using Dev-Core_To_Dev-LoadTimes (reversed)

Change 3214981 on 2016/11/30 by Gil.Gribb

	UE4 - Make sure that subobjects of CDOs are exported even if they don't have any direct references; they might be archetypes.

Change 3215392 on 2016/11/30 by Steve.Robb

	Error out on editor builds when reading a non-boolean value in an archive.

Change 3215674 on 2016/11/30 by Steve.Robb

	Replacement of a custom scope-exit setup with our standard macro.

Change 3215720 on 2016/11/30 by Steve.Robb

	Default constructor for TTuple which value-initializes the elements.

Change 3216777 on 2016/12/01 by Graeme.Thornton

	Add NoRedist and NotForLicensees folders in the game folder to the C# version of config file hierarchy.

Change 3216858 on 2016/12/01 by Graeme.Thornton

	Improvements to pak signing and encryption
	 - Remove compile time defines for encryption key and signing keys
	 - Embed keys in executable by including in the UBT generated UELinkerFixups.cpp file
	 - Add key access core delegate for passing these keys through to pak platform file

Change 3216860 on 2016/12/01 by Graeme.Thornton

	Re-enable pak signing and encryption in ShooterGame

Change 3216861 on 2016/12/01 by Graeme.Thornton

	Re-enable pak signing and encryption in Paragon

Change 3217076 on 2016/12/01 by Gil.Gribb

	UE4 - replaced !GIsIntialLoad with EVENT_DRIVEN_ASYNC_LOAD_ACTIVE_AT_RUNTIME as a first step toward boot time EDL

Change 3221139 on 2016/12/05 by Ben.Woodhouse

	Dummy integrate of CL 3221131 (versioning workarounds) from Dev-LoadTimes4.14 with accept target to prevent those workarounds being merged out of dev-loadtimes4.14.

	Command:
	p4 integrate //UE4/Dev-LoadTimes4.14/...@3221131,3221131

	w/ resolve/accept-target

Change 3221410 on 2016/12/05 by Steve.Robb

	Allow Algo::IsSorted to work on C arrays and initializer_lists.

Change 3221673 on 2016/12/05 by Gil.Gribb

	set up an intentional merge conflict so we can do the right thing when dev-BP and dev-core come together

Change 3223182 on 2016/12/06 by Chris.Wood

	Add EngineModeEx to CrashReportCommon
	[UE-39258] - Update CrashReporter to include whether Engine crashes are from vanilla build or not

Change 3224646 on 2016/12/07 by Robert.Manuszewski

	It's no longer necessary to rebuild the cooked exe to enable the event driven loader. EDL and the new async IO are now controlled with ini settings only.

Change 3224647 on 2016/12/07 by Robert.Manuszewski

	Event Driven Loader is now enabled by default.

Change 3224669 on 2016/12/07 by Chris.Wood

	Compressing some crash files output from CRP to S3 (CRP v1.2.11)
	[UE-37564] - Add compression support for CRP crash files in S3

	Add EngineModeEx to support to CRP (CRP v1.2.10)
	[UE-39258] - Update CrashReporter to include whether Engine crashes are from vanilla build or not

Change 3224873 on 2016/12/07 by Robert.Manuszewski

	A few fixes to the EDL macro refactor.

Change 3224933 on 2016/12/07 by Robert.Manuszewski

	Async Loading settings will now be logged on startup in cooked games

Change 3224984 on 2016/12/07 by Robert.Manuszewski

	Small fix to EDL and new async IO global var init.

Change 3225027 on 2016/12/07 by Robert.Manuszewski

	Re-enabling EDL in ShooterGame since it no longer requires the cooked exe to be re-compiled (and EDL is on by default anyway).

Change 3226702 on 2016/12/08 by Steve.Robb

	UHT can now skip over macro-like identifiers while parsing.
	Correctly handles FTypefaceEntry::Name parsing, which was broken by the workaround in CL# 3219332.
	Syntax workaround reverted in FTypefaceEntry.

	#jira UE-38231

Change 3226756 on 2016/12/08 by Steve.Robb

	LexicalConversion renamed to Lex after #core discussion.
	Lex::ToString overloads added for FString itself.

Change 3226766 on 2016/12/08 by Steve.Robb

	FCookStatsManager::ToString replaced with Lex::ToString, as it now supports the required functionality.

Change 3226949 on 2016/12/08 by Robert.Manuszewski

	Fixing static analysis warnings

Change 3228566 on 2016/12/09 by Robert.Manuszewski

	Making UBT not think it needs to use separate temporary target for non source code projects when EDL settings don't match default.

Change 3228601 on 2016/12/09 by Steve.Robb

	Fix for TSparseArray visualization of types smaller than 8 bytes.

Change 3228654 on 2016/12/09 by Gil.Gribb

	UE4 - Fix very old bug with lock free lists, probably not relevant to anything we currently use.

Change 3228697 on 2016/12/09 by Graeme.Thornton

	Rebuilt UnrealPak

[CL 3228932 by Robert Manuszewski in Main branch]
2016-12-09 11:36:14 -05:00

340 lines
9.9 KiB
C++

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "GenericPlatform/GenericPlatformFile.h"
#include "NetworkMessage.h"
#include "ServerTOC.h"
class FScopedEvent;
DECLARE_LOG_CATEGORY_EXTERN(LogNetworkPlatformFile, Log, All);
/**
* Wrapper to redirect the low level file system to a server
*/
class NETWORKFILE_API FNetworkPlatformFile : public IPlatformFile
{
friend class FAsyncFileSync;
friend void ReadUnsolicitedFile(int32 InNumUnsolictedFiles, FNetworkPlatformFile& InNetworkFile, IPlatformFile& InInnerPlatformFile, FString& InServerEngineDir, FString& InServerGameDir);
/**
* Initialize network platform file give the specified host IP
*
* @param Inner Inner platform file
* @param HostIP host IP address
* @return true if the initialization succeeded, false otherwise
*/
virtual bool InitializeInternal(IPlatformFile* Inner, const TCHAR* HostIP);
public:
static const TCHAR* GetTypeName()
{
return TEXT("NetworkFile");
}
/** Constructor */
FNetworkPlatformFile();
/** Destructor */
virtual ~FNetworkPlatformFile();
virtual bool ShouldBeUsed(IPlatformFile* Inner, const TCHAR* CmdLine) const override;
virtual bool Initialize(IPlatformFile* Inner, const TCHAR* CmdLine) override;
virtual void InitializeAfterSetActive() override;
virtual IPlatformFile* GetLowerLevel() override
{
return InnerPlatformFile;
}
virtual void SetLowerLevel(IPlatformFile* NewLowerLevel) override
{
InnerPlatformFile = NewLowerLevel;
}
virtual void GetTimeStampPair(const TCHAR* PathA, const TCHAR* PathB, FDateTime& OutTimeStampA, FDateTime& OutTimeStampB)
{
OutTimeStampA = GetTimeStamp(PathA);
OutTimeStampB = GetTimeStamp(PathB);
if (GetLowerLevel() && OutTimeStampA == FDateTime::MinValue() && OutTimeStampB == FDateTime::MinValue())
{
GetLowerLevel()->GetTimeStampPair(PathA, PathB, OutTimeStampA, OutTimeStampB);
}
}
virtual const TCHAR* GetName() const override
{
return FNetworkPlatformFile::GetTypeName();
}
virtual bool IsUsable()
{
return bIsUsable;
}
virtual bool FileExists(const TCHAR* Filename) override
{
FFileInfo Info;
GetFileInfo(Filename, Info);
return Info.FileExists;
}
virtual int64 FileSize(const TCHAR* Filename) override
{
FFileInfo Info;
GetFileInfo(Filename, Info);
return Info.Size;
}
virtual bool DeleteFile(const TCHAR* Filename) override;
virtual bool IsReadOnly(const TCHAR* Filename) override
{
FFileInfo Info;
GetFileInfo(Filename, Info);
return Info.ReadOnly;
}
virtual bool MoveFile(const TCHAR* To, const TCHAR* From) override;
virtual bool SetReadOnly(const TCHAR* Filename, bool bNewReadOnlyValue) override;
virtual FDateTime GetTimeStamp(const TCHAR* Filename) override
{
FFileInfo Info;
GetFileInfo(Filename, Info);
return Info.TimeStamp;
}
virtual void SetTimeStamp(const TCHAR* Filename, FDateTime DateTime) override;
virtual FDateTime GetAccessTimeStamp(const TCHAR* Filename) override
{
FFileInfo Info;
GetFileInfo(Filename, Info);
return Info.AccessTimeStamp;
}
virtual FString GetFilenameOnDisk(const TCHAR* Filename) override
{
return Filename;
}
virtual IFileHandle* OpenRead(const TCHAR* Filename, bool bAllowWrite = false) override;
virtual IFileHandle* OpenWrite(const TCHAR* Filename, bool bAppend = false, bool bAllowRead = false) override;
virtual bool DirectoryExists(const TCHAR* Directory) override;
virtual bool CreateDirectoryTree(const TCHAR* Directory) override;
virtual bool CreateDirectory(const TCHAR* Directory) override;
virtual bool DeleteDirectory(const TCHAR* Directory) override;
virtual FFileStatData GetStatData(const TCHAR* FilenameOrDirectory) override;
virtual bool IterateDirectory(const TCHAR* Directory, IPlatformFile::FDirectoryVisitor& Visitor) override;
virtual bool IterateDirectoryRecursively(const TCHAR* Directory, IPlatformFile::FDirectoryVisitor& Visitor) override;
virtual bool IterateDirectoryStat(const TCHAR* Directory, IPlatformFile::FDirectoryStatVisitor& Visitor) override;
virtual bool IterateDirectoryStatRecursively(const TCHAR* Directory, IPlatformFile::FDirectoryStatVisitor& Visitor) override;
virtual bool DeleteDirectoryRecursively(const TCHAR* Directory) override;
virtual bool CopyFile(const TCHAR* To, const TCHAR* From, EPlatformFileRead ReadFlags = EPlatformFileRead::None, EPlatformFileWrite WriteFlags = EPlatformFileWrite::None) override;
virtual FString ConvertToAbsolutePathForExternalAppForRead( const TCHAR* Filename ) override;
virtual FString ConvertToAbsolutePathForExternalAppForWrite( const TCHAR* Filename ) override;
virtual bool SendMessageToServer(const TCHAR* Message, IPlatformFile::IFileServerMessageHandler* Handler) override;
virtual bool SendPayloadAndReceiveResponse(TArray<uint8>& In, TArray<uint8>& Out);
virtual bool ReceiveResponse(TArray<uint8>& Out);
bool SendReadMessage(uint8* Destination, int64 BytesToRead);
bool SendWriteMessage(const uint8* Source, int64 BytesToWrite);
static void ConvertServerFilenameToClientFilename(FString& FilenameToConvert, const FString& InServerEngineDir, const FString& InServerGameDir);
protected:
/**
* Send a heartbeat message to the file server. This will tell it we are alive, as well as
* get back a list of files that have been updated on the server (so we can toss our copy)
*/
virtual void PerformHeartbeat();
virtual void GetFileInfo(const TCHAR* Filename, FFileInfo& Info);
/**
* Convert the given filename from the server to the client version of it
* NOTE: Potentially modifies the input FString!!!!
*
* @param FilenameToConvert Upon input, the server version of the filename. After the call, the client version
*/
virtual void ConvertServerFilenameToClientFilename(FString& FilenameToConvert);
virtual void FillGetFileList(FNetworkFileArchive& Payload, bool bInStreamingFileRequest);
virtual void ProcessServerInitialResponse(FArrayReader& InResponse, int32 OutServerPackageVersion, int32 OutServerPackageLicenseeVersion);
private:
/**
* Returns whether the passed in extension is a video
* extension. Extensions with and without trailing dots are supported.
*
* @param Extension to test.
* @return True if Ext is a video extension. e.g. .mp4
*/
static bool IsMediaExtension(const TCHAR* Ext);
/**
* Returns whether the passed in extension is a an additional (but non asset) cooked file.
*
* @param Extension to test.
* @return True if Ext is a additional cooked file. e.g. .ubulk, .ufont
*/
static bool IsAdditionalCookedFileExtension(const TCHAR* Ext);
/**
* @return true if the path exists in a directory that should always use the local filesystem
* This version does not worry about initialization or thread safety, do not call directly
*/
bool IsInLocalDirectoryUnGuarded(const FString& Filename);
/**
* @return true if the path exists in a directory that should always use the local filesystem
*/
bool IsInLocalDirectory(const FString& Filename);
/**
* Given a filename, make sure the file exists on the local filesystem
*/
void EnsureFileIsLocal(const FString& Filename);
protected:
/**
* Does normal path standardization, and also any extra modifications to make string comparisons against
* the internal directory list work properly.
*/
void MakeStandardNetworkFilename(FString& Filename);
protected:
/** This is true after the DDC directories have been loaded from the DDC system */
bool bHasLoadedDDCDirectories;
/** The file interface to read/write local files with */
IPlatformFile* InnerPlatformFile;
/** This keeps track of what files have been "EnsureFileIsLocal'd" */
TSet<FString> CachedLocalFiles;
/** The server engine dir */
FString ServerEngineDir;
/** The server game dir */
FString ServerGameDir;
/** This is the "TOC" of the server */
FServerTOC ServerFiles;
/** Set of directories that should use the local filesystem */
TArray<FString> LocalDirectories;
FCriticalSection SynchronizationObject;
FCriticalSection LocalDirectoriesCriticalSection;
bool bIsUsable;
int32 FileServerPort;
private:
/* Unsolicitied files events */
FScopedEvent *FinishedAsyncNetworkReadUnsolicitedFiles;
FScopedEvent *FinishedAsyncWriteUnsolicitedFiles;
// Our network Transport.
class ITransport* Transport;
static FString MP4Extension;
static FString BulkFileExtension;
static FString ExpFileExtension;
static FString FontFileExtension;
};
class SOCKETS_API FNetworkFileHandle : public IFileHandle
{
FNetworkPlatformFile& Network;
FString Filename;
int64 FilePos;
int64 Size;
bool bWritable;
bool bReadable;
public:
FNetworkFileHandle(FNetworkPlatformFile& InNetwork, const TCHAR* InFilename, int64 InFilePos, int64 InFileSize, bool bWriting)
: Network(InNetwork)
, Filename(InFilename)
, FilePos(InFilePos)
, Size(InFileSize)
, bWritable(bWriting)
, bReadable(!bWriting)
{
}
virtual int64 Tell() override
{
return FilePos;
}
virtual bool Seek(int64 NewPosition) override
{
if (NewPosition >= 0 && NewPosition <= Size)
{
FilePos = NewPosition;
return true;
}
return false;
}
virtual bool SeekFromEnd(int64 NewPositionRelativeToEnd = 0) override
{
return Seek(Size + NewPositionRelativeToEnd);
}
virtual bool Read(uint8* Destination, int64 BytesToRead) override
{
bool Result = false;
if (bReadable && BytesToRead >= 0 && BytesToRead + FilePos <= Size)
{
if (BytesToRead == 0)
{
Result = true;
}
else
{
Result = Network.SendReadMessage(Destination, BytesToRead);
if (Result)
{
FilePos += BytesToRead;
}
}
}
return Result;
}
virtual bool Write(const uint8* Source, int64 BytesToWrite) override
{
bool Result = false;
if (bWritable && BytesToWrite >= 0)
{
if (BytesToWrite == 0)
{
Result = true;
}
else
{
Result = Network.SendWriteMessage(Source, BytesToWrite);
if (Result)
{
FilePos += BytesToWrite;
Size = FMath::Max<int64>(FilePos, Size);
}
}
}
return Result;
}
};