Files
UnrealEngineUWP/Engine/Source/Runtime/NetworkFile/Private/NetworkPlatformFile.cpp
Matt Kuhlenschmidt 47a097348a Copying //UE4/Dev-Editor to //UE4/Dev-Main (Source: //UE4/Dev-Editor @ 3167359)
#lockdown Nick.Penwarden
#rb none

==========================
MAJOR FEATURES + CHANGES
==========================

Change 3152124 on 2016/10/05 by Jamie.Dale

	Fixed SOutputLog filter not handling OnTextCommitted

Change 3152255 on 2016/10/05 by Michael.Dupuis

	#jira UE-28173 Support \" properly in FName

Change 3152273 on 2016/10/05 by Nick.Darnell

	Core - The module manager is now thread safer, we had a critical section around the internal module list - but we were incrementing/decrementing shared pointers to module data shared pointers that were not thread safe outside of the critical section.  Ran into a crash working on some heavily threaded code in automation.

Change 3152314 on 2016/10/05 by Nick.Darnell

	Automation - Continued work to rough out the automation workflow for screenshot.  Still lots of work remaining, but it appears the basic of approving images might be working as of this CL.

Change 3152316 on 2016/10/05 by Michael.Dupuis

	#jira UE-30346 Update selection when in tree view mode

Change 3152317 on 2016/10/05 by Nick.Darnell

	Automation - Adding some test shots to compare against to EngineTest for screenshot approval.

Change 3152319 on 2016/10/05 by Michael.Dupuis

	#jira UE-29817 StringAssetReference will now only open an Asset picker (not actor picker) as the goal is to reference an asset

Change 3152521 on 2016/10/05 by Nick.Darnell

	Automation - Fixing some issues with where it reads the screenshot compare rules.

Change 3152536 on 2016/10/05 by Alexis.Matte

	Fix FBX automation test.
	- Make sure the fbx test can avoid automatic detection of the mesh type
	- Avoid to log the warning when the importer set the material usage after creating a material for skeletalmesh.

Change 3152572 on 2016/10/05 by Nick.Darnell

	Automation - The GameProjectAutomationTests now do some pre-run house cleaning to make sure the project doesn't already exist, and tries to remove it if it was created previously but not deleted.

Change 3152591 on 2016/10/05 by Nick.Darnell

	Automation - Changing the game project errors to be errors.

Change 3153115 on 2016/10/06 by Jamie.Dale

	Removed superflous padding when SPropertyEditorAsset had no buttons

Change 3153215 on 2016/10/06 by Michael.Dupuis

	Fixed build warning

Change 3153248 on 2016/10/06 by Nick.Darnell

	Automation - Working on solving projects not being generated, suspect UBT isn't built or isn't available.

Change 3153255 on 2016/10/06 by Nick.Darnell

	PR #2835: Fix TestEqual AddError Message in FAutomationTestBase (Contributed by dorgonman)

	#jira UE-36922

Change 3153300 on 2016/10/06 by Nick.Darnell

	Automation - Enabled verbose logging to automation build farm.

Change 3153343 on 2016/10/06 by Matt.Kuhlenschmidt

	PR #2825: More project launcher progress improvements (Contributed by projectgheist)

Change 3153506 on 2016/10/06 by Gareth.Martin

	Fixed crash trying to edit landscape with r.LightPropagationVolume=1 enabled
	#jira UE-36933

Change 3153752 on 2016/10/06 by tim.gautier

	Add toggle button to UMG_Behavior. Set Level Blueprint for TM-UMG to AllWidget

Change 3153763 on 2016/10/06 by Nick.Darnell

	Automation - Disable verbose logging.

Change 3153778 on 2016/10/06 by Nick.Darnell

	PR #2837: Fixed engine compilation with defined LOG_SLATE_EVENTS (Contributed by Pierdek)

	#jira UE-36940

Change 3153943 on 2016/10/06 by Nick.Darnell

	Automation - Disabling some broken tests.

Change 3154035 on 2016/10/06 by Nick.Darnell

	Automation - Fixing re-runs for tests that want them.  Previously this wasn't working for any test that was run using the Reprostring method of being executed.

Change 3154039 on 2016/10/06 by Nick.Darnell

	Automation - Updating some test assets in the EngineTest project.

Change 3154476 on 2016/10/07 by Richard.TalbotWatkin

	Fix to regression where vertex color view in Mesh Paint Mode no longer worked correctly. We now allow selected static meshes to go down the dynamic path if VertexColors show mode is active.
	#jira UE-36772 - Selecting a channel in Vertex Paint mode does not show the channel color

Change 3154650 on 2016/10/07 by Alexis.Matte

	Add new front axis facing X option to fbx importer

Change 3154785 on 2016/10/07 by Nick.Darnell

	Automation - Continued work on automation, ripping out the message bus from the screenshot manager, that's causing the screenshot manager to recieve shots from every machine ever running tests.  The Automation Controller is now in charge, and will tell the screenshot manager whatever it needs.

Change 3155131 on 2016/10/07 by Michael.Dupuis

	#jira UE-36509 Do not disabled inverse filter when doing a sync to asset

Change 3155141 on 2016/10/07 by Michael.Dupuis

	#jira UE-36056 Do not open the Actor Picker if we're working on an archetype object

Change 3155262 on 2016/10/07 by Michael.Dupuis

	#jira UE-19737 reset ctrl key when resetting state to None

Change 3156326 on 2016/10/09 by Matt.Kuhlenschmidt

	Fixed crash when asset picker is used without a property editor (usually a heavily customized property).

Change 3156473 on 2016/10/10 by Richard.TalbotWatkin

	Attempt to make geometry render / rebuild more robust in the hope of catching UE-36265.
	#jira UE-36265 - [CrashReport] UE4Editor_Engine!FModelSceneProxy::HasSelectedSurfaces() [modelrender.cpp:538]

Change 3156479 on 2016/10/10 by Richard.TalbotWatkin

	Fixed non-editor build.

Change 3156579 on 2016/10/10 by Alexis.Matte

	Add a check to make sure curve pointer is valid.
	#jira UE-36177

Change 3156585 on 2016/10/10 by Ben.Marsh

	Fix line endings for screenshot settings.

Change 3156617 on 2016/10/10 by Matt.Kuhlenschmidt

	Disable per-pixel blending of menus by default.  Causes artifacts on windows versions and we are not using it.

Change 3156674 on 2016/10/10 by Nick.Darnell

	Automation - Continued work on the automation workflow.  Now screenshot functional test actors actually have the screenshot processed for differences during the test back on the test controller machine, and then waits for the results of the comparison to be performed.

Change 3156709 on 2016/10/10 by Alexis.Matte

	#jira UE-16337
	Make sure the base mesh import data transform is used when we import a LOD.

Change 3156714 on 2016/10/10 by Nick.Darnell

	Automation - Fixing -game crash due to TestName being null in functional test.

Change 3156721 on 2016/10/10 by Nick.Darnell

	Automation - FunctionalAITest now implements IsReady to check if the navigation mesh has finished being built.

Change 3156748 on 2016/10/10 by Nick.Darnell

	Autopmation - Fixing a warning.

Change 3156943 on 2016/10/10 by Alex.Delesky

	Fixed an issue where closing out the "Add Code" window with an active toast stating that an IDE was downloading would prevent the toast from clearing correctly.

	#jira none

Change 3156946 on 2016/10/10 by Alex.Delesky

	#jira UE-36414 - Adding support for the P4Changelist command line argument to the P4 operations that were missing it.

Change 3158215 on 2016/10/11 by Nick.Darnell

	Automation - Continued work on the screenshot comparison, fixed a crash, the system now reports if the screenshot was new, as well as similar, so that the script can react and warn when new screenshots are produced.  Manually fired screenshots now properly wait until they've been compared before the test moves forward.

Change 3158322 on 2016/10/11 by Michael.Dupuis

	#jira UE-36063 If the collection is not shown when trying to save one, force show them so the user understand what is going on

Change 3158333 on 2016/10/11 by Alex.Delesky

	#jira UE-36829 - Rebuilt SVN 1.9.4 libs for Windows with CyrusSASL 2.1.26 and SWIG 3.0.10 support.

Change 3158399 on 2016/10/11 by Nick.Darnell

	Automation - TTF Font log statements that were not warnings are no longer warnings.

Change 3158406 on 2016/10/11 by Nick.Darnell

	Automation - Updating some automation scripts in the engnine that were using file paths to now use FStringAssetReferences, that cleaned up a lot of nasty convert filepath to asset reference code in the tests.  Also introducing some maps, and setting up the configs to use them to try and appease some of the existing tests that were expecting them.

Change 3158419 on 2016/10/11 by Alex.Delesky

	#jira UE-36829 - SVN 1.9.4 libraries, but also with Java 8u101 support.

Change 3158537 on 2016/10/11 by Nick.Darnell

	Automation - Updating some automation scripts in the engnine that were using file paths to now use FStringAssetReferences, that cleaned up a lot of nasty convert filepath to asset reference code in the tests.  Also introducing some maps, and setting up the configs to use them to try and appease some of the existing tests that were expecting them.

	Adding some missing files.

Change 3158726 on 2016/10/11 by Michael.Dupuis

	#jira UE-37001 Perform manual migration of UICurve to proper config category

Change 3158728 on 2016/10/11 by Nick.Darnell

	Automation - Fixing some warnings.  Adding more testing to the Domino map to serve as a better example.

Change 3158753 on 2016/10/11 by Michael.Dupuis

	#jira UE-26261 change it's by its

Change 3158984 on 2016/10/11 by Alexis.Matte

	Fix D&D folder import in content browser. We have to expand the root directory to have the correct path.
	#jira UE-32155

Change 3159640 on 2016/10/12 by Jamie.Dale

	Split localized package redirection out of FCoreDelegates::PackageNameResolvers

	They're different enough in behavior that the delegate resolution was breaking the localized package resolution by resolving in too many places and causing the localized package to be loaded with its real localized name as well as the fake non-localized name.

	#jira UE-37119

Change 3159741 on 2016/10/12 by Nick.Darnell

	Slate - Fixing the SGraphPanel's mouse offset calculations so that it works with HDPI mode.

Change 3159762 on 2016/10/12 by Nick.Darnell

	Automation - Adding a way to take a screenshot with the UI, not great yet - it doesn't really obey the rules of resolution, because of the method it uses.

Change 3160210 on 2016/10/12 by Gareth.Martin

	Fixed crash when changing Landscape LOD while using "Grass use Landscape Lightmap"

Change 3160216 on 2016/10/12 by Gareth.Martin

	Removed unused FLandscapeComponentSceneProxy::LayerNames and made LayerColors editor-only
	Fixed negative LODBias on landscape components to actually do anything

Change 3160239 on 2016/10/12 by Gareth.Martin

	Removed an unused variable

Change 3160455 on 2016/10/12 by Jamie.Dale

	Fixed FText properties exported to asset tags displaying in their NSLOCTEXT form in the asset tooltips

Change 3160457 on 2016/10/12 by Jamie.Dale

	Localization automation now groups everything into a single CL and reverts PO files without significant changes

Change 3160554 on 2016/10/12 by Nick.Darnell

	UMG - Fixing some panning logic to work with HDPI mode in the designer.

Change 3161712 on 2016/10/13 by Jamie.Dale

	Fixed TSharedMapView using hard-coded types

Change 3163044 on 2016/10/14 by Jamie.Dale

	Fixed line-break iterators incorrectly breaking words in CJK

Change 3163046 on 2016/10/14 by Jamie.Dale

	Text layout no longer creates break candidates when wrapping is disabled

Change 3163217 on 2016/10/14 by Jamie.Dale

	Fixed ensure caused by FMediaTextureResource::GetResourceSizeEx

Change 3163641 on 2016/10/14 by Alex.Delesky

	#jira UE-36829 - Updated Mac SVN libraries, along with a more accurate changelog for Windows SVN libs

Change 3164428 on 2016/10/17 by Nick.Darnell

	Slate - Making the FReply for SetMousePos easier to debug, since that option is potentially very frustrating to debug when someone is changing it.

Change 3164833 on 2016/10/17 by Jamie.Dale

	Fixed ParseNumber consuming strings with multiple sequential dots as valid numbers, eg) "10..."

Change 3164868 on 2016/10/17 by Alexis.Matte

	Remove re-import material and LOD import material
	#jira UE-36640

Change 3164874 on 2016/10/17 by Alexis.Matte

	Fix fbx scene re-import of staticmesh loosing there materials
	#jira UE-37032

Change 3165080 on 2016/10/17 by Alexis.Matte

	Remove skinxx workflow for static mesh
	#jira UE-37262

Change 3165232 on 2016/10/17 by Nick.Darnell

	Automation - Adding some sub-level testing.

Change 3165822 on 2016/10/18 by Nick.Darnell

	Slate - Add a counter to track how much time we spend drawing custom verts each frame.

Change 3165934 on 2016/10/18 by Nick.Darnell

	Slate - Addding cycle counters to SInvalidationPanel's Tick and Paint.

Change 3165947 on 2016/10/18 by Nick.Darnell

	Slate - Addding very verbose slate stats behind a compiler flag to avoid the large overhead of these stats.  To enable them you'll need to set WITH_VERY_VERBOSE_SLATE_STATS to 1, added a guide on debugging slate performance to the SlateGlobals.h

	// HOW TO GET AN IN-DEPTH PERFORMANCE ANALYSIS OF SLATE
	//
	// Step 1)
	//    Set WITH_VERY_VERBOSE_SLATE_STATS to 1.
	//
	// Step 2)
	//    When running the game (outside of the editor), run these commandline options
	//    in order and you'll get a large dump of where all the time is going in Slate.
	//
	//    stat group enable slateverbose
	//    stat group enable slateveryverbose
	//    stat dumpave -root=stat_slate -num=120 -ms=0

Change 3165962 on 2016/10/18 by Nick.Darnell

	UMG - Play first frame of sequence in UMG immediately when told to play an animation.

Change 3165981 on 2016/10/18 by Nick.Darnell

	Core - GetDisplayNameText now stores the FName for DisplayName in a static instead of using TEXT("DisplayName").

Change 3166000 on 2016/10/18 by Jamie.Dale

	Removed bulk-data from fonts

	The main complaints about composite fonts have always been:
	 1) They use too much memory at runtime.
	 2) They bloat if you use the same font face twice.
	 3) They often break when used outside the game thread.

	This change aims to address all of these issues by removing bulk-data from fonts (which was the cause of the memory bloat and threading issues), and introduces UFontFace assets (which allow you to re-use the same font face in different entries within a composite font).

	No existing font data is lost during this transition, however you must manually update your UFont assets to reference external UFontFace assets before you're able to edit the existing data (which is automatically upgraded to UFontFace assets *within* the UFont package). This upgrade can be done via the composite font editor.

	During cook these UFontFace assets write out the actual TTF/OTF font data as a .ufont file, which is what FreeType actually loads at runtime (the internals of the Slate font cache are now completely independent of UObjects and their lifetimes and loading concerns).

	Since we're now always loading files from disk, we can also give the user an option of whether to "pre-load" (which will load the entire font into memory, like bulk-data always used to), or "stream" the font from disk (which has minimal memory overhead, but needs decent I/O performance).

Change 3166001 on 2016/10/18 by Jamie.Dale

	Updated the Launcher to no longer use bulk-data for fonts

Change 3166003 on 2016/10/18 by Jamie.Dale

	Updated the Engine fonts to use UFontFace assets

Change 3166028 on 2016/10/18 by Alex.Delesky

	#jira UE-37021 - The scrollbar will now remain at the bottom of the output log after specifying a filter.

Change 3166071 on 2016/10/18 by Nick.Darnell

	Slate - Fixing a warning about hiding an inherited member.

Change 3166213 on 2016/10/18 by Jamie.Dale

	Fixing crash caused by accessing a zeroed FText

Change 3166222 on 2016/10/18 by Nick.Darnell

	Automation - Adding some code to end the sub level test when it starts.

Change 3166231 on 2016/10/18 by Nick.Darnell

	Editor - Fixing the build warning, SOutputLog.cpp:748:4: warning: field 'TextLayout' will be initialized after field 'CachedNumMessages'

Change 3166717 on 2016/10/18 by Nick.Darnell

	Automation - Removing references to old options that are no longer file paths, and are now StringAssetReferences these options were not being used by anyone as far as I can tell.

	#jira UE-37482

Change 3167279 on 2016/10/19 by Jamie.Dale

	Fixed text render component regression with custom MIDs

	#jira UE-37305

Change 3167356 on 2016/10/19 by Alexis.Matte

	Make sure the old asset are build correctly
	#jira UE-37461

Change 3167359 on 2016/10/19 by Alexis.Matte

	Fix re-import of mesh material assignment regression
	#jira UE-37479

[CL 3168049 by Matt Kuhlenschmidt in Main branch]
2016-10-19 15:01:48 -04:00

1309 lines
42 KiB
C++

// Copyright 1998-2016 Epic Games, Inc. All Rights Reserved.
#include "NetworkFilePrivatePCH.h"
#include "NetworkPlatformFile.h"
#include "MultichannelTCP.h"
#include "DerivedDataCacheInterface.h"
#include "PackageName.h"
#include "HTTPTransport.h"
#include "TCPTransport.h"
#if WITH_UNREAL_DEVELOPER_TOOLS
#include "Developer/PackageDependencyInfo/Public/PackageDependencyInfo.h"
#endif //WITH_UNREAL_DEVELOPER_TOOLS
#include "IPlatformFileModule.h"
DEFINE_LOG_CATEGORY(LogNetworkPlatformFile);
FString FNetworkPlatformFile::MP4Extension = TEXT(".mp4");
FString FNetworkPlatformFile::BulkFileExtension = TEXT(".ubulk");
FString FNetworkPlatformFile::FontFileExtension = TEXT(".ufont");
FNetworkPlatformFile::FNetworkPlatformFile()
: bHasLoadedDDCDirectories(false)
, InnerPlatformFile(NULL)
, bIsUsable(false)
, FinishedAsyncNetworkReadUnsolicitedFiles(NULL)
, FinishedAsyncWriteUnsolicitedFiles(NULL)
, Transport(NULL)
{
}
bool FNetworkPlatformFile::ShouldBeUsed(IPlatformFile* Inner, const TCHAR* CmdLine) const
{
FString HostIp;
return FParse::Value(CmdLine, TEXT("-FileHostIP="), HostIp);
}
ITransport *CreateTransportForHostAddress(const FString &HostIp )
{
if ( HostIp.StartsWith(TEXT("tcp://")))
{
return new FTCPTransport();
}
if ( HostIp.StartsWith(TEXT("http://")))
{
#if ENABLE_HTTP_FOR_NFS
return new FHTTPTransport();
#endif
}
// no transport specified assuming tcp
return new FTCPTransport();
}
bool FNetworkPlatformFile::Initialize(IPlatformFile* Inner, const TCHAR* CmdLine)
{
bool bResult = false;
FString HostIpString;
if (FParse::Value(CmdLine, TEXT("-FileHostIP="), HostIpString))
{
TArray<FString> HostIpList;
if (HostIpString.ParseIntoArray(HostIpList, TEXT("+"), true) > 0)
{
for (int32 HostIpIndex = 0; !bResult && HostIpIndex < HostIpList.Num(); ++HostIpIndex)
{
// Try to initialize with each of the IP addresses found in the command line until we
// get a working one.
// find the correct transport for this ip address
Transport = CreateTransportForHostAddress( HostIpList[HostIpIndex] );
UE_LOG(LogNetworkPlatformFile, Warning, TEXT("Created transport for %s."), *HostIpList[HostIpIndex]);
if ( Transport )
{
bResult = Transport->Initialize( *HostIpList[HostIpIndex] ) && InitializeInternal(Inner, *HostIpList[HostIpIndex]);
if (bResult)
break;
UE_LOG(LogNetworkPlatformFile, Warning, TEXT("Failed to initialize %s."), *HostIpList[HostIpIndex]);
// try a different host might be a different protocol
delete Transport;
}
Transport = NULL;
}
}
}
return bResult;
}
bool FNetworkPlatformFile::InitializeInternal(IPlatformFile* Inner, const TCHAR* HostIP)
{
// This platform file requires an inner.
check(Inner != NULL);
InnerPlatformFile = Inner;
if (HostIP == NULL)
{
UE_LOG(LogNetworkPlatformFile, Error, TEXT("No Host IP specified in the commandline."));
bIsUsable = false;
return false;
}
// Save and Intermediate directories are always local
LocalDirectories.Add(FPaths::EngineDir() / TEXT("Binaries"));
LocalDirectories.Add(FPaths::EngineIntermediateDir());
LocalDirectories.Add(FPaths::GameDir() / TEXT("Binaries"));
LocalDirectories.Add(FPaths::GameIntermediateDir());
LocalDirectories.Add(FPaths::GameSavedDir() / TEXT("Backup"));
LocalDirectories.Add(FPaths::GameSavedDir() / TEXT("Config"));
LocalDirectories.Add(FPaths::GameSavedDir() / TEXT("Logs"));
LocalDirectories.Add(FPaths::GameSavedDir() / TEXT("Sandboxes"));
if (InnerPlatformFile->GetLowerLevel())
{
InnerPlatformFile->GetLowerLevel()->AddLocalDirectories(LocalDirectories);
}
else
{
InnerPlatformFile->AddLocalDirectories(LocalDirectories);
}
FNetworkFileArchive Payload(NFS_Messages::Heartbeat);
FArrayReader Out;
if (!SendPayloadAndReceiveResponse(Payload,Out))
bIsUsable = true;
// lets see we can test whether the server is up.
if (Out.Num())
{
FCommandLine::AddToSubprocessCommandline( *FString::Printf( TEXT("-FileHostIP=%s"), HostIP ) );
bIsUsable = true;
}
return bIsUsable;
}
bool FNetworkPlatformFile::SendPayloadAndReceiveResponse(TArray<uint8>& In, TArray<uint8>& Out)
{
if ( FinishedAsyncNetworkReadUnsolicitedFiles )
{
delete FinishedAsyncNetworkReadUnsolicitedFiles;
FinishedAsyncNetworkReadUnsolicitedFiles = NULL;
}
return Transport->SendPayloadAndReceiveResponse( In, Out );
}
bool FNetworkPlatformFile::ReceiveResponse(TArray<uint8> &Out )
{
return Transport->ReceiveResponse( Out );
}
void FNetworkPlatformFile::InitializeAfterSetActive()
{
double NetworkFileStartupTime = 0.0;
{
SCOPE_SECONDS_COUNTER(NetworkFileStartupTime);
// send the filenames and timestamps to the server
FNetworkFileArchive Payload(NFS_Messages::GetFileList);
FillGetFileList(Payload, false);
// send the directories over, and wait for a response
FArrayReader Response;
if (!SendPayloadAndReceiveResponse(Payload, Response))
{
delete Transport;
return;
}
else
{
// receive the cooked version information
int32 ServerPackageVersion = 0;
int32 ServerPackageLicenseeVersion = 0;
ProcessServerInitialResponse(Response, ServerPackageVersion, ServerPackageLicenseeVersion);
/* The server root content directories */
TArray<FString> ServerRootContentDirectories;
Response << ServerRootContentDirectories;
// receive a list of the cache files and their timestamps
TMap<FString, FDateTime> ServerCachedFiles;
Response << ServerCachedFiles;
bool bDeleteAllFiles = true;
// Check the stored cooked version
FString CookedVersionFile = FPaths::GeneratedConfigDir() / TEXT("CookedVersion.txt");
if (InnerPlatformFile->FileExists(*CookedVersionFile) == true)
{
IFileHandle* FileHandle = InnerPlatformFile->OpenRead(*CookedVersionFile);
if (FileHandle != NULL)
{
int32 StoredPackageCookedVersion;
int32 StoredPackageCookedLicenseeVersion;
if (FileHandle->Read((uint8*)&StoredPackageCookedVersion, sizeof(int32)) == true)
{
if (FileHandle->Read((uint8*)&StoredPackageCookedLicenseeVersion, sizeof(int32)) == true)
{
if ((ServerPackageVersion == StoredPackageCookedVersion) &&
(ServerPackageLicenseeVersion == StoredPackageCookedLicenseeVersion))
{
bDeleteAllFiles = false;
}
else
{
UE_LOG(LogNetworkPlatformFile, Display,
TEXT("Engine version mismatch: Server %d.%d, Stored %d.%d\n"),
ServerPackageVersion, ServerPackageLicenseeVersion,
StoredPackageCookedVersion, StoredPackageCookedLicenseeVersion);
}
}
}
delete FileHandle;
}
}
else
{
UE_LOG(LogNetworkPlatformFile, Display, TEXT("Cooked version file missing: %s\n"), *CookedVersionFile);
}
if (bDeleteAllFiles == true)
{
// Make sure the config file exists...
InnerPlatformFile->CreateDirectoryTree(*(FPaths::GeneratedConfigDir()));
// Update the cooked version file
IFileHandle* FileHandle = InnerPlatformFile->OpenWrite(*CookedVersionFile);
if (FileHandle != NULL)
{
FileHandle->Write((const uint8*)&ServerPackageVersion, sizeof(int32));
FileHandle->Write((const uint8*)&ServerPackageLicenseeVersion, sizeof(int32));
delete FileHandle;
}
}
// list of directories to skip
TArray<FString> DirectoriesToSkip;
TArray<FString> DirectoriesToNotRecurse;
// use the timestamp grabbing visitor to get all the content times
FLocalTimestampDirectoryVisitor Visitor(*InnerPlatformFile, DirectoriesToSkip, DirectoriesToNotRecurse, false);
/*TArray<FString> RootContentPaths;
FPackageName::QueryRootContentPaths(RootContentPaths); */
for (TArray<FString>::TConstIterator RootPathIt(ServerRootContentDirectories); RootPathIt; ++RootPathIt)
{
/*const FString& RootPath = *RootPathIt;
const FString& ContentFolder = FPackageName::LongPackageNameToFilename(RootPath);*/
const FString& ContentFolder = *RootPathIt;
InnerPlatformFile->IterateDirectory( *ContentFolder, Visitor);
}
// delete out of date files using the server cached files
for (TMap<FString, FDateTime>::TIterator It(ServerCachedFiles); It; ++It)
{
bool bDeleteFile = bDeleteAllFiles;
FString ServerFile = It.Key();
// Convert the filename to the client version
ConvertServerFilenameToClientFilename(ServerFile);
// Set it in the visitor file times list
// If there is any pathing difference (relative path, or whatever) between the server's filelist and the results
// of platform directory iteration then this will Add a new entry rather than override the existing one. This causes local file deletes
// and longer loads as we will never see the benefits of local device caching.
Visitor.FileTimes.Add(ServerFile, FDateTime::MinValue());
if (bDeleteFile == false)
{
// Check the time stamps...
// get local time
FDateTime LocalTime = InnerPlatformFile->GetTimeStamp(*ServerFile);
// If local time == MinValue than the file does not exist in the cache.
if (LocalTime != FDateTime::MinValue())
{
FDateTime ServerTime = It.Value();
// delete if out of date
// We will use 1.0 second as the tolerance to cover any platform differences in resolution
FTimespan TimeDiff = LocalTime - ServerTime;
double TimeDiffInSeconds = TimeDiff.GetTotalSeconds();
bDeleteFile = (TimeDiffInSeconds > 1.0) || (TimeDiffInSeconds < -1.0);
if (bDeleteFile == true)
{
if (InnerPlatformFile->FileExists(*ServerFile) == true)
{
UE_LOG(LogNetworkPlatformFile, Display, TEXT("Deleting cached file: TimeDiff %5.3f, %s"), TimeDiffInSeconds, *It.Key());
}
else
{
// It's a directory
bDeleteFile = false;
}
}
else
{
UE_LOG(LogNetworkPlatformFile, Display, TEXT("Keeping cached file: %s, TimeDiff worked out ok"), *ServerFile);
}
}
}
if (bDeleteFile == true)
{
InnerPlatformFile->DeleteFile(*ServerFile);
}
}
// Any content files we have locally that were not cached, delete them
for (TMap<FString, FDateTime>::TIterator It(Visitor.FileTimes); It; ++It)
{
if (It.Value() != FDateTime::MinValue())
{
// This was *not* found in the server file list... delete it
UE_LOG(LogNetworkPlatformFile, Display, TEXT("Deleting cached file: %s"), *It.Key());
InnerPlatformFile->DeleteFile(*It.Key());
}
}
// make sure we can sync a file
FString TestSyncFile = FPaths::Combine(*(FPaths::EngineDir()), TEXT("Config/BaseEngine.ini"));
InnerPlatformFile->SetReadOnly(*TestSyncFile, false);
InnerPlatformFile->DeleteFile(*TestSyncFile);
if (InnerPlatformFile->FileExists(*TestSyncFile))
{
UE_LOG(LogNetworkPlatformFile, Fatal, TEXT("Could not delete file sync test file %s."), *TestSyncFile);
}
EnsureFileIsLocal(TestSyncFile);
if (!InnerPlatformFile->FileExists(*TestSyncFile) || InnerPlatformFile->FileSize(*TestSyncFile) < 1)
{
UE_LOG(LogNetworkPlatformFile, Fatal, TEXT("Could not sync test file %s."), *TestSyncFile);
}
}
}
FPlatformMisc::LowLevelOutputDebugStringf(TEXT("Network file startup time: %5.3f seconds\n"), NetworkFileStartupTime);
}
FNetworkPlatformFile::~FNetworkPlatformFile()
{
if (!GIsRequestingExit) // the socket subsystem is probably already gone, so it will crash if we clean up
{
if ( FinishedAsyncNetworkReadUnsolicitedFiles )
{
delete FinishedAsyncNetworkReadUnsolicitedFiles; // wait here for any async unsolicited files to finish reading being read from the network
FinishedAsyncNetworkReadUnsolicitedFiles = NULL;
}
if ( FinishedAsyncWriteUnsolicitedFiles )
{
delete FinishedAsyncWriteUnsolicitedFiles; // wait here for any async unsolicited files to finish writing
FinishedAsyncWriteUnsolicitedFiles = NULL;
}
delete Transport; // close our sockets.
}
}
bool FNetworkPlatformFile::DeleteFile(const TCHAR* Filename)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// make and send payload (this is how we would do for sending all commands over the network)
// FNetworkFileArchive Payload(NFS_Messages::DeleteFile);
// Payload << Filename;
// return FNFSMessageHeader::WrapAndSendPayload(Payload, FileSocket);
// perform a local operation
return InnerPlatformFile->DeleteFile(Filename);
}
bool FNetworkPlatformFile::MoveFile(const TCHAR* To, const TCHAR* From)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// make and send payload (this is how we would do for sending all commands over the network)
// FNetworkFileArchive Payload(NFS_Messages::MoveFile);
// Payload << To << From;
// return FNFSMessageHeader::WrapAndSendPayload(Payload, FileSocket);
FString RelativeFrom = From;
MakeStandardNetworkFilename(RelativeFrom);
// don't copy files in local directories
if (!IsInLocalDirectory(RelativeFrom))
{
// make sure the source file exists here
EnsureFileIsLocal(RelativeFrom);
}
// perform a local operation
return InnerPlatformFile->MoveFile(To, From);
}
bool FNetworkPlatformFile::SetReadOnly(const TCHAR* Filename, bool bNewReadOnlyValue)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// perform a local operation
return InnerPlatformFile->SetReadOnly(Filename, bNewReadOnlyValue);
}
void FNetworkPlatformFile::SetTimeStamp(const TCHAR* Filename, FDateTime DateTime)
{
// perform a local operation
InnerPlatformFile->SetTimeStamp(Filename, DateTime);
}
IFileHandle* FNetworkPlatformFile::OpenRead(const TCHAR* Filename, bool bAllowWrite)
{
// FScopeLock ScopeLock(&SynchronizationObject);
FString RelativeFilename = Filename;
MakeStandardNetworkFilename(RelativeFilename);
// don't copy files in local directories
if (!IsInLocalDirectory(RelativeFilename))
{
EnsureFileIsLocal(RelativeFilename);
}
double StartTime;
float ThisTime;
StartTime = FPlatformTime::Seconds();
IFileHandle* Result = InnerPlatformFile->OpenRead(Filename, bAllowWrite);
ThisTime = 1000.0f * float(FPlatformTime::Seconds() - StartTime);
//UE_LOG(LogNetworkPlatformFile, Display, TEXT("Open local file %6.2fms"), ThisTime);
return Result;
}
IFileHandle* FNetworkPlatformFile::OpenWrite(const TCHAR* Filename, bool bAppend, bool bAllowRead)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// just let the physical file interface write the file (we don't write over the network)
return InnerPlatformFile->OpenWrite(Filename, bAppend, bAllowRead);
}
bool FNetworkPlatformFile::CreateDirectoryTree(const TCHAR* Directory)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// perform a local operation
return InnerPlatformFile->CreateDirectoryTree(Directory);
}
bool FNetworkPlatformFile::CreateDirectory(const TCHAR* Directory)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// perform a local operation
return InnerPlatformFile->CreateDirectory(Directory);
}
bool FNetworkPlatformFile::DeleteDirectory(const TCHAR* Directory)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// perform a local operation
return InnerPlatformFile->DeleteDirectory(Directory);
}
FFileStatData FNetworkPlatformFile::GetStatData(const TCHAR* FilenameOrDirectory)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// perform a local operation
return InnerPlatformFile->GetStatData(FilenameOrDirectory);
}
bool FNetworkPlatformFile::IterateDirectory(const TCHAR* InDirectory, IPlatformFile::FDirectoryVisitor& Visitor)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// for .dll, etc searches that don't specify a path, we need to strip off the path
// before we send it to the visitor
bool bHadNoPath = InDirectory[0] == 0;
// local files go right to the source
FString RelativeDirectory = InDirectory;
MakeStandardNetworkFilename(RelativeDirectory);
if (IsInLocalDirectory(RelativeDirectory))
{
return InnerPlatformFile->IterateDirectory(InDirectory, Visitor);
}
// we loop until this is false
bool RetVal = true;
FServerTOC::FDirectory* ServerDirectory = ServerFiles.FindDirectory(RelativeDirectory);
if (ServerDirectory != NULL)
{
// loop over the server files and look if they are in this exact directory
for (FServerTOC::FDirectory::TIterator It(*ServerDirectory); It && RetVal == true; ++It)
{
if (FPaths::GetPath(It.Key()) == RelativeDirectory)
{
// timestamps of 0 mean directories
bool bIsDirectory = It.Value() == 0;
// visit (stripping off the path if needed)
RetVal = Visitor.Visit(bHadNoPath ? *FPaths::GetCleanFilename(It.Key()) : *It.Key(), bIsDirectory);
}
}
}
return RetVal;
}
bool FNetworkPlatformFile::IterateDirectoryRecursively(const TCHAR* InDirectory, IPlatformFile::FDirectoryVisitor& Visitor)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// local files go right to the source
FString RelativeDirectory = InDirectory;
MakeStandardNetworkFilename(RelativeDirectory);
if (IsInLocalDirectory(RelativeDirectory))
{
return InnerPlatformFile->IterateDirectoryRecursively(InDirectory, Visitor);
}
// we loop until this is false
bool RetVal = true;
for (TMap<FString, FServerTOC::FDirectory*>::TIterator DirIt(ServerFiles.Directories); DirIt && RetVal == true; ++DirIt)
{
if (DirIt.Key().StartsWith(RelativeDirectory))
{
FServerTOC::FDirectory& ServerDirectory = *DirIt.Value();
// loop over the server files and look if they are in this exact directory
for (FServerTOC::FDirectory::TIterator It(ServerDirectory); It && RetVal == true; ++It)
{
// timestamps of 0 mean directories
bool bIsDirectory = It.Value() == 0;
// visit!
RetVal = Visitor.Visit(*It.Key(), bIsDirectory);
}
}
}
return RetVal;
}
bool FNetworkPlatformFile::IterateDirectoryStat(const TCHAR* InDirectory, IPlatformFile::FDirectoryStatVisitor& Visitor)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// for .dll, etc searches that don't specify a path, we need to strip off the path
// before we send it to the visitor
bool bHadNoPath = InDirectory[0] == 0;
// local files go right to the source
FString RelativeDirectory = InDirectory;
MakeStandardNetworkFilename(RelativeDirectory);
if (IsInLocalDirectory(RelativeDirectory))
{
return InnerPlatformFile->IterateDirectoryStat(InDirectory, Visitor);
}
// we loop until this is false
bool RetVal = true;
FServerTOC::FDirectory* ServerDirectory = ServerFiles.FindDirectory(RelativeDirectory);
if (ServerDirectory != NULL)
{
// loop over the server files and look if they are in this exact directory
for (FServerTOC::FDirectory::TIterator It(*ServerDirectory); It && RetVal == true; ++It)
{
if (FPaths::GetPath(It.Key()) == RelativeDirectory)
{
// timestamps of 0 mean directories
bool bIsDirectory = It.Value() == 0;
// todo: this data is just wrong for most things, but can we afford to get the files from the server to get the correct info? Could the server provide this instead?
const FFileStatData StatData(
FDateTime::MinValue(),
FDateTime::MinValue(),
(bIsDirectory) ? FDateTime::MinValue() : It.Value(),
-1, // FileSize
bIsDirectory,
true // IsReadOnly
);
// visit (stripping off the path if needed)
RetVal = Visitor.Visit(bHadNoPath ? *FPaths::GetCleanFilename(It.Key()) : *It.Key(), StatData);
}
}
}
return RetVal;
}
bool FNetworkPlatformFile::IterateDirectoryStatRecursively(const TCHAR* InDirectory, IPlatformFile::FDirectoryStatVisitor& Visitor)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// local files go right to the source
FString RelativeDirectory = InDirectory;
MakeStandardNetworkFilename(RelativeDirectory);
if (IsInLocalDirectory(RelativeDirectory))
{
return InnerPlatformFile->IterateDirectoryStatRecursively(InDirectory, Visitor);
}
// we loop until this is false
bool RetVal = true;
for (TMap<FString, FServerTOC::FDirectory*>::TIterator DirIt(ServerFiles.Directories); DirIt && RetVal == true; ++DirIt)
{
if (DirIt.Key().StartsWith(RelativeDirectory))
{
FServerTOC::FDirectory& ServerDirectory = *DirIt.Value();
// loop over the server files and look if they are in this exact directory
for (FServerTOC::FDirectory::TIterator It(ServerDirectory); It && RetVal == true; ++It)
{
// timestamps of 0 mean directories
bool bIsDirectory = It.Value() == 0;
// todo: this data is just wrong for most things, but can we afford to get the files from the server to get the correct info? Could the server provide this instead?
const FFileStatData StatData(
FDateTime::MinValue(),
FDateTime::MinValue(),
(bIsDirectory) ? FDateTime::MinValue() : It.Value(),
0, // FileSize
bIsDirectory,
true // IsReadOnly
);
// visit!
RetVal = Visitor.Visit(*It.Key(), StatData);
}
}
}
return RetVal;
}
bool FNetworkPlatformFile::DeleteDirectoryRecursively(const TCHAR* Directory)
{
// FScopeLock ScopeLock(&SynchronizationObject);
// perform a local operation
return InnerPlatformFile->DeleteDirectory(Directory);
}
bool FNetworkPlatformFile::CopyFile(const TCHAR* To, const TCHAR* From, EPlatformFileRead ReadFlags, EPlatformFileWrite WriteFlags)
{
// FScopeLock ScopeLock(&SynchronizationObject);
FString RelativeFrom = From;
MakeStandardNetworkFilename(RelativeFrom);
// don't copy files in local directories
if (!IsInLocalDirectory(RelativeFrom))
{
// make sure the source file exists here
EnsureFileIsLocal(RelativeFrom);
}
// perform a local operation
return InnerPlatformFile->CopyFile(To, From, ReadFlags, WriteFlags);
}
FString FNetworkPlatformFile::ConvertToAbsolutePathForExternalAppForRead( const TCHAR* Filename )
{
FString RelativeFrom = Filename;
MakeStandardNetworkFilename(RelativeFrom);
if (!IsInLocalDirectory(RelativeFrom))
{
EnsureFileIsLocal(RelativeFrom);
}
return InnerPlatformFile->ConvertToAbsolutePathForExternalAppForRead(Filename);
}
FString FNetworkPlatformFile::ConvertToAbsolutePathForExternalAppForWrite( const TCHAR* Filename )
{
FString RelativeFrom = Filename;
MakeStandardNetworkFilename(RelativeFrom);
if (!IsInLocalDirectory(RelativeFrom))
{
EnsureFileIsLocal(RelativeFrom);
}
return InnerPlatformFile->ConvertToAbsolutePathForExternalAppForWrite(Filename);
}
bool FNetworkPlatformFile::DirectoryExists(const TCHAR* Directory)
{
if (InnerPlatformFile->DirectoryExists(Directory))
{
return true;
}
// If there are any syncable files in this directory, consider it existing
FString RelativeDirectory = Directory;
MakeStandardNetworkFilename(RelativeDirectory);
FServerTOC::FDirectory* ServerDirectory = ServerFiles.FindDirectory(RelativeDirectory);
return ServerDirectory != NULL;
}
void FNetworkPlatformFile::GetFileInfo(const TCHAR* Filename, FFileInfo& Info)
{
FString RelativeFilename = Filename;
MakeStandardNetworkFilename(RelativeFilename);
// don't copy files in local directories
if (!IsInLocalDirectory(RelativeFilename))
{
EnsureFileIsLocal(RelativeFilename);
}
const FFileStatData StatData = InnerPlatformFile->GetStatData(Filename);
Info.FileExists = StatData.bIsValid && !StatData.bIsDirectory;
Info.ReadOnly = StatData.bIsReadOnly;
Info.Size = StatData.FileSize;
Info.TimeStamp = StatData.ModificationTime;
Info.AccessTimeStamp = StatData.AccessTime;
}
void FNetworkPlatformFile::ConvertServerFilenameToClientFilename(FString& FilenameToConvert)
{
FNetworkPlatformFile::ConvertServerFilenameToClientFilename(FilenameToConvert, ServerEngineDir, ServerGameDir);
}
void FNetworkPlatformFile::FillGetFileList(FNetworkFileArchive& Payload, bool bInStreamingFileRequest)
{
TArray<FString> TargetPlatformNames;
FPlatformMisc::GetValidTargetPlatforms(TargetPlatformNames);
FString GameName = FApp::GetGameName();
if (FPaths::IsProjectFilePathSet())
{
GameName = FPaths::GetProjectFilePath();
}
FString EngineRelPath = FPaths::EngineDir();
FString EngineRelPluginPath = FPaths::EnginePluginsDir();
FString GameRelPath = FPaths::GameDir();
FString GameRelPluginPath = FPaths::GamePluginsDir();
TArray<FString> Directories;
Directories.Add(EngineRelPath);
Directories.Add(EngineRelPluginPath);
Directories.Add(GameRelPath);
Directories.Add(GameRelPluginPath);
Payload << TargetPlatformNames;
Payload << GameName;
Payload << EngineRelPath;
Payload << GameRelPath;
Payload << Directories;
Payload << bInStreamingFileRequest;
}
void FNetworkPlatformFile::ProcessServerInitialResponse(FArrayReader& InResponse, int32 OutServerPackageVersion, int32 OutServerPackageLicenseeVersion)
{
// Receive the cooked version information.
InResponse << OutServerPackageVersion;
InResponse << OutServerPackageLicenseeVersion;
// receive the server engine and game dir
InResponse << ServerEngineDir;
InResponse << ServerGameDir;
UE_LOG(LogNetworkPlatformFile, Display, TEXT(" Server EngineDir = %s"), *ServerEngineDir);
UE_LOG(LogNetworkPlatformFile, Display, TEXT(" Local EngineDir = %s"), *FPaths::EngineDir());
UE_LOG(LogNetworkPlatformFile, Display, TEXT(" Server GameDir = %s"), *ServerGameDir);
UE_LOG(LogNetworkPlatformFile, Display, TEXT(" Local GameDir = %s"), *FPaths::GameDir());
// Receive a list of files and their timestamps.
TMap<FString, FDateTime> ServerFileMap;
InResponse << ServerFileMap;
for (TMap<FString, FDateTime>::TIterator It(ServerFileMap); It; ++It)
{
FString ServerFile = It.Key();
ConvertServerFilenameToClientFilename(ServerFile);
ServerFiles.AddFileOrDirectory(ServerFile, It.Value());
}
}
bool FNetworkPlatformFile::SendReadMessage(uint8* Destination, int64 BytesToRead)
{
// FScopeLock ScopeLock(&SynchronizationObject);
return true;
}
bool FNetworkPlatformFile::SendWriteMessage(const uint8* Source, int64 BytesToWrite)
{
// FScopeLock ScopeLock(&SynchronizationObject);
return true;
}
bool FNetworkPlatformFile::SendMessageToServer(const TCHAR* Message, IPlatformFile::IFileServerMessageHandler* Handler)
{
// handle the recompile shaders message
// @todo: Maybe we should just send the string message to the server, but then we'd have to
// handle the return from the server in a generic way
if (FCString::Stricmp(Message, TEXT("RecompileShaders")) == 0)
{
FNetworkFileArchive Payload(NFS_Messages::RecompileShaders);
// let the handler fill out the object
Handler->FillPayload(Payload);
FArrayReader Response;
if (!SendPayloadAndReceiveResponse(Payload, Response))
{
return false;
}
// locally delete any files that were modified on the server, so that any read will recache the file
// this has to be done in this class, not in the Handler (which can't access these members)
TArray<FString> ModifiedFiles;
Response << ModifiedFiles;
if( InnerPlatformFile != NULL )
{
for (int32 Index = 0; Index < ModifiedFiles.Num(); Index++)
{
InnerPlatformFile->DeleteFile(*ModifiedFiles[Index]);
CachedLocalFiles.Remove(ModifiedFiles[Index]);
ServerFiles.AddFileOrDirectory(ModifiedFiles[Index], FDateTime::UtcNow());
}
}
// let the handler process the response directly
Handler->ProcessResponse(Response);
}
return true;
}
static FThreadSafeCounter OutstandingAsyncWrites;
class FAsyncNetworkWriteWorker : public FNonAbandonableTask
{
public:
/** Filename To write to**/
FString Filename;
/** An archive to read the file contents from */
FArchive* FileArchive;
/** timestamp for the file **/
FDateTime ServerTimeStamp;
IPlatformFile& InnerPlatformFile;
FScopedEvent* Event;
uint8 Buffer[128 * 1024];
/** Constructor
*/
FAsyncNetworkWriteWorker(const TCHAR* InFilename, FArchive* InArchive, FDateTime InServerTimeStamp, IPlatformFile* InInnerPlatformFile, FScopedEvent* InEvent)
: Filename(InFilename)
, FileArchive(InArchive)
, ServerTimeStamp(InServerTimeStamp)
, InnerPlatformFile(*InInnerPlatformFile)
, Event(InEvent)
{
}
/** Write the file */
void DoWork()
{
if (InnerPlatformFile.FileExists(*Filename))
{
InnerPlatformFile.SetReadOnly(*Filename, false);
InnerPlatformFile.DeleteFile(*Filename);
}
// Read FileSize first so that the correct amount of data is read from the archive
// before exiting this worker.
uint64 FileSize;
*FileArchive << FileSize;
if (ServerTimeStamp != FDateTime::MinValue()) // if the file didn't actually exist on the server, don't create a zero byte file
{
FString TempFilename = Filename + TEXT(".tmp");
InnerPlatformFile.CreateDirectoryTree(*FPaths::GetPath(Filename));
{
TAutoPtr<IFileHandle> FileHandle;
FileHandle = InnerPlatformFile.OpenWrite(*TempFilename);
if (!FileHandle)
{
UE_LOG(LogNetworkPlatformFile, Fatal, TEXT("Could not open file for writing '%s'."), *TempFilename);
}
// now write the file from bytes pulled from the archive
// read/write a chunk at a time
uint64 RemainingData = FileSize;
while (RemainingData)
{
// read next chunk from archive
uint32 LocalSize = FPlatformMath::Min<uint32>(ARRAY_COUNT(Buffer), RemainingData);
FileArchive->Serialize(Buffer, LocalSize);
// write it out
if (!FileHandle->Write(Buffer, LocalSize))
{
UE_LOG(LogNetworkPlatformFile, Fatal, TEXT("Could not write '%s'."), *TempFilename);
}
// decrement how much is left
RemainingData -= LocalSize;
}
// delete async write archives
if (Event)
{
delete FileArchive;
}
if (InnerPlatformFile.FileSize(*TempFilename) != FileSize)
{
UE_LOG(LogNetworkPlatformFile, Fatal, TEXT("Did not write '%s'."), *TempFilename);
}
}
// rename from temp filename to real filename
InnerPlatformFile.MoveFile(*Filename, *TempFilename);
// now set the server's timestamp on the local file (so we can make valid comparisons)
InnerPlatformFile.SetTimeStamp(*Filename, ServerTimeStamp);
FDateTime CheckTime = InnerPlatformFile.GetTimeStamp(*Filename);
if (CheckTime < ServerTimeStamp)
{
UE_LOG(LogNetworkPlatformFile, Fatal, TEXT("Could Not Set Timestamp '%s'."), *Filename);
}
}
if (Event)
{
if (OutstandingAsyncWrites.Decrement() == 0)
{
Event->Trigger(); // last file, fire trigger
}
}
}
FORCEINLINE TStatId GetStatId() const
{
return TStatId();
//RETURN_QUICK_DECLARE_CYCLE_STAT(FAsyncNetworkWriteWorker, STATGROUP_ThreadPoolAsyncTasks);
}
};
/**
* Write a file async or sync, with the data coming from a TArray or an FArchive/Filesize
*/
void SyncWriteFile(FArchive* Archive, const FString& Filename, FDateTime ServerTimeStamp, IPlatformFile& InnerPlatformFile)
{
FScopedEvent* NullEvent = NULL;
(new FAutoDeleteAsyncTask<FAsyncNetworkWriteWorker>(*Filename, Archive, ServerTimeStamp, &InnerPlatformFile, NullEvent))->StartSynchronousTask();
}
void AsyncWriteFile(FArchive* Archive, const FString& Filename, FDateTime ServerTimeStamp, IPlatformFile& InnerPlatformFile, FScopedEvent* Event = NULL)
{
(new FAutoDeleteAsyncTask<FAsyncNetworkWriteWorker>(*Filename, Archive, ServerTimeStamp, &InnerPlatformFile, Event))->StartBackgroundTask();
}
void AsyncReadUnsolicitedFiles(int32 InNumUnsolictedFiles, FNetworkPlatformFile& InNetworkFile, IPlatformFile& InInnerPlatformFile, FString& InServerEngineDir, FString& InServerGameDir, FScopedEvent *InNetworkDoneEvent, FScopedEvent *InWritingDoneEvent)
{
class FAsyncReadUnsolicitedFile : public FNonAbandonableTask
{
public:
int32 NumUnsolictedFiles;
FNetworkPlatformFile& NetworkFile;
IPlatformFile& InnerPlatformFile;
FString ServerEngineDir;
FString ServerGameDir;
FScopedEvent* NetworkDoneEvent; // finished using the network
FScopedEvent* WritingDoneEvent; // finished writing the files to disk
FAsyncReadUnsolicitedFile(int32 In_NumUnsolictedFiles, FNetworkPlatformFile* In_NetworkFile, IPlatformFile* In_InnerPlatformFile, FString& In_ServerEngineDir, FString& In_ServerGameDir, FScopedEvent *In_NetworkDoneEvent, FScopedEvent *In_WritingDoneEvent )
: NumUnsolictedFiles(In_NumUnsolictedFiles)
, NetworkFile(*In_NetworkFile)
, InnerPlatformFile(*In_InnerPlatformFile)
, ServerEngineDir(In_ServerEngineDir)
, ServerGameDir(In_ServerGameDir)
, NetworkDoneEvent(In_NetworkDoneEvent)
, WritingDoneEvent(In_WritingDoneEvent)
{
}
/** Write the file */
void DoWork()
{
OutstandingAsyncWrites.Add( NumUnsolictedFiles );
for (int32 Index = 0; Index < NumUnsolictedFiles; Index++)
{
FArrayReader* UnsolictedResponse = new FArrayReader;
if (!NetworkFile.ReceiveResponse(*UnsolictedResponse))
{
UE_LOG(LogNetworkPlatformFile, Fatal, TEXT("Receive failure!"));
return;
}
FString UnsolictedReplyFile;
*UnsolictedResponse << UnsolictedReplyFile;
if (!UnsolictedReplyFile.IsEmpty())
{
FNetworkPlatformFile::ConvertServerFilenameToClientFilename(UnsolictedReplyFile, ServerEngineDir, ServerGameDir);
// get the server file timestamp
FDateTime UnsolictedServerTimeStamp;
*UnsolictedResponse << UnsolictedServerTimeStamp;
// write the file by pulling out of the FArrayReader
AsyncWriteFile(UnsolictedResponse, UnsolictedReplyFile, UnsolictedServerTimeStamp, InnerPlatformFile, WritingDoneEvent);
}
}
NetworkDoneEvent->Trigger();
}
FORCEINLINE TStatId GetStatId() const
{
RETURN_QUICK_DECLARE_CYCLE_STAT(FAsyncReadUnsolicitedFile, STATGROUP_ThreadPoolAsyncTasks);
}
};
(new FAutoDeleteAsyncTask<FAsyncReadUnsolicitedFile>(InNumUnsolictedFiles, &InNetworkFile, &InInnerPlatformFile, InServerEngineDir, InServerGameDir, InNetworkDoneEvent, InWritingDoneEvent))->StartSynchronousTask();
}
bool FNetworkPlatformFile::IsMediaExtension(const TCHAR* Ext)
{
if (*Ext != TEXT('.'))
{
return MP4Extension.EndsWith(Ext);
}
else
{
return MP4Extension == Ext;
}
}
bool FNetworkPlatformFile::IsAdditionalCookedFileExtension(const TCHAR* Ext)
{
if (*Ext != TEXT('.'))
{
return BulkFileExtension.EndsWith(Ext) || FontFileExtension.EndsWith(Ext);
}
else
{
return BulkFileExtension == Ext || FontFileExtension == Ext;
}
}
/**
* Given a filename, make sure the file exists on the local filesystem
*/
void FNetworkPlatformFile::EnsureFileIsLocal(const FString& Filename)
{
double StartTime;
float ThisTime;
StartTime = FPlatformTime::Seconds();
{
FScopeLock ScopeLock(&SynchronizationObject);
// have we already cached this file?
if (CachedLocalFiles.Find(Filename) != NULL)
{
return;
}
}
if ( FinishedAsyncNetworkReadUnsolicitedFiles )
{
delete FinishedAsyncNetworkReadUnsolicitedFiles; // wait here for any async unsolicited files to finish reading being read from the network
FinishedAsyncNetworkReadUnsolicitedFiles = NULL;
}
if( FinishedAsyncWriteUnsolicitedFiles)
{
delete FinishedAsyncWriteUnsolicitedFiles; // wait here for any async unsolicited files to finish writing to disk
FinishedAsyncWriteUnsolicitedFiles = NULL;
}
FScopeLock ScopeLock(&SynchronizationObject);
ThisTime = 1000.0f * float(FPlatformTime::Seconds() - StartTime);
//UE_LOG(LogNetworkPlatformFile, Display, TEXT("Lock and wait for old async writes %6.2fms"), ThisTime);
// have we already cached this file? (test again, since some other thread might have done this between waits)
if (CachedLocalFiles.Find(Filename) != NULL)
{
return;
}
// even if an error occurs later, we still want to remember not to try again
CachedLocalFiles.Add(Filename);
StartTime = FPlatformTime::Seconds();
// no need to read it if it already exists
// @todo: Handshake with server to delete files that are out of date
if (InnerPlatformFile->FileExists(*Filename))
{
return;
}
ThisTime = 1000.0f * float(FPlatformTime::Seconds() - StartTime);
//UE_LOG(LogNetworkPlatformFile, Display, TEXT("Check for local file %6.2fms - %s"), ThisTime, *Filename);
// this is a bit of a waste if we aren't doing cook on the fly, but we assume missing asset files are relatively rare
FString Extension = FPaths::GetExtension(Filename, true);
bool bIsCookable = GConfig && GConfig->IsReadyForUse() && (FPackageName::IsPackageExtension(*Extension) || IsMediaExtension(*Extension) || IsAdditionalCookedFileExtension(*Extension));
// we only copy files that actually exist on the server, can greatly reduce network traffic for, say,
// the INT file each package tries to load
if (!bIsCookable && (ServerFiles.FindFile(Filename) == NULL))
{
// Uncomment this to have the server file list dumped
// the first time a file requested is not found.
#if 0
static bool sb_DumpedServer = false;
if (sb_DumpedServer == false)
{
FPlatformMisc::LowLevelOutputDebugStringf(TEXT("Dumping server files... %s not found\n"), *Filename);
for (TMap<FString, FServerTOC::FDirectory*>::TIterator ServerDumpIt(ServerFiles.Directories); ServerDumpIt; ++ServerDumpIt)
{
FServerTOC::FDirectory& Directory = *ServerDumpIt.Value();
for (FServerTOC::FDirectory::TIterator DirDumpIt(Directory); DirDumpIt; ++DirDumpIt)
{
FPlatformMisc::LowLevelOutputDebugStringf(TEXT("%10s - %s\n"), *(DirDumpIt.Value().ToString()), *(DirDumpIt.Key()));
}
}
sb_DumpedServer = true;
}
#endif
return;
}
// send the filename over (cast away const here because we know this << will not modify the string)
FNetworkFileArchive Payload(NFS_Messages::SyncFile);
Payload << (FString&)Filename;
StartTime = FPlatformTime::Seconds();
// allocate array reader on the heap, because the SyncWriteFile function will delete it
FArrayReader Response;
if (!SendPayloadAndReceiveResponse(Payload, Response))
{
UE_LOG(LogNetworkPlatformFile, Fatal, TEXT("Receive failure!"));
return;
}
ThisTime = 1000.0f * float(FPlatformTime::Seconds() - StartTime);
//UE_LOG(LogNetworkPlatformFile, Display, TEXT("Send and receive %6.2fms"), ThisTime);
StartTime = FPlatformTime::Seconds();
FString ReplyFile;
Response << ReplyFile;
ConvertServerFilenameToClientFilename(ReplyFile);
check(ReplyFile == Filename);
// get the server file timestamp
FDateTime ServerTimeStamp;
Response << ServerTimeStamp;
// write the file in chunks, synchronously
SyncWriteFile(&Response, ReplyFile, ServerTimeStamp, *InnerPlatformFile);
int32 NumUnsolictedFiles;
Response << NumUnsolictedFiles;
if (NumUnsolictedFiles)
{
check( FinishedAsyncNetworkReadUnsolicitedFiles == NULL );
check( FinishedAsyncWriteUnsolicitedFiles == NULL );
FinishedAsyncNetworkReadUnsolicitedFiles = new FScopedEvent;
FinishedAsyncWriteUnsolicitedFiles = new FScopedEvent;
AsyncReadUnsolicitedFiles(NumUnsolictedFiles, *this, *InnerPlatformFile, ServerEngineDir, ServerGameDir, FinishedAsyncNetworkReadUnsolicitedFiles, FinishedAsyncWriteUnsolicitedFiles);
}
ThisTime = 1000.0f * float(FPlatformTime::Seconds() - StartTime);
//UE_LOG(LogNetworkPlatformFile, Display, TEXT("Write file to local %6.2fms"), ThisTime);
}
static FString NetworkPlatformFileEndChop(TEXT("/"));
void FNetworkPlatformFile::MakeStandardNetworkFilename(FString& Filename)
{
FPaths::MakeStandardFilename(Filename);
Filename.RemoveFromEnd(NetworkPlatformFileEndChop, ESearchCase::CaseSensitive);
}
bool FNetworkPlatformFile::IsInLocalDirectoryUnGuarded(const FString& Filename)
{
// cache the directory of the input file
FString Directory = FPaths::GetPath(Filename);
// look if the file is in a local directory
for (int32 DirIndex = 0; DirIndex < LocalDirectories.Num(); DirIndex++)
{
if (Directory.StartsWith(LocalDirectories[DirIndex]))
{
return true;
}
}
// if not local, talk to the server
return false;
}
bool FNetworkPlatformFile::IsInLocalDirectory(const FString& Filename)
{
if (!bHasLoadedDDCDirectories)
{
// need to be careful here to avoid initializing the DDC from the wrong thread or using LocalDirectories while it is being initialized
FScopeLock ScopeLock(&LocalDirectoriesCriticalSection);
if (IsInGameThread() && GConfig && GConfig->IsReadyForUse())
{
// one time DDC directory initialization
// add any DDC directories to our list of local directories (local = inner platform file, it may
// actually live on a server, but it will use the platform's file system)
if (GetDerivedDataCache())
{
TArray<FString> DdcDirectories;
GetDerivedDataCacheRef().GetDirectories(DdcDirectories);
LocalDirectories.Append(DdcDirectories);
}
FPlatformMisc::MemoryBarrier();
bHasLoadedDDCDirectories = true;
}
return IsInLocalDirectoryUnGuarded(Filename);
}
// once the DDC is initialized, we don't need to lock a critical section anymore
return IsInLocalDirectoryUnGuarded(Filename);
}
void FNetworkPlatformFile::PerformHeartbeat()
{
// send the filename over (cast away const here because we know this << will not modify the string)
FNetworkFileArchive Payload(NFS_Messages::Heartbeat);
// send the filename over
FArrayReader Response;
if (!SendPayloadAndReceiveResponse(Payload, Response))
{
return;
}
// get any files that have been modified on the server -
TArray<FString> UpdatedFiles;
Response << UpdatedFiles;
// delete any outdated files from the client
// @todo: This may need a critical section around all calls to LowLevel in the other functions
// because we don't want to delete files while other threads are using them!
for (int32 FileIndex = 0; FileIndex < UpdatedFiles.Num(); FileIndex++)
{
UE_LOG(LogNetworkPlatformFile, Log, TEXT("Server updated file '%s', deleting local copy"), *UpdatedFiles[FileIndex]);
if (InnerPlatformFile->DeleteFile(*UpdatedFiles[FileIndex]) == false)
{
UE_LOG(LogNetworkPlatformFile, Error, TEXT("Failed to delete %s, someone is probably accessing without FNetworkPlatformFile, or we need better thread protection"), *UpdatedFiles[FileIndex]);
}
}
}
void FNetworkPlatformFile::ConvertServerFilenameToClientFilename(FString& FilenameToConvert, const FString& InServerEngineDir, const FString& InServerGameDir)
{
if (FilenameToConvert.StartsWith(InServerEngineDir))
{
FilenameToConvert = FilenameToConvert.Replace(*InServerEngineDir, *(FPaths::EngineDir()));
}
else if (FilenameToConvert.StartsWith(InServerGameDir))
{
FilenameToConvert = FilenameToConvert.Replace(*InServerGameDir, *(FPaths::GameDir()));
}
}
/**
* Module for the network file
*/
class FNetworkFileModule : public IPlatformFileModule
{
public:
virtual IPlatformFile* GetPlatformFile() override
{
static TScopedPointer<IPlatformFile> AutoDestroySingleton(new FNetworkPlatformFile());
return AutoDestroySingleton.GetOwnedPointer();
}
};
IMPLEMENT_MODULE(FNetworkFileModule, NetworkFile);