Files
UnrealEngineUWP/Engine/Source/Programs/AutomationTool/BuildGraph/LockFile.cs
Luke Thatcher 0d4895e8e7 Copying //UE4/Dev-Console to //UE4/Dev-Main (Source: //UE4/Dev-Console @ 4070461)
#lockdown Nick.Penwarden
#rb none

============================
  MAJOR FEATURES & CHANGES
============================

Change 3751378 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3748735 to //UE4/Dev-Console/...

Change 3807818 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3803271 to //UE4/Dev-Console/...

Change 3818577 by Ben.Woodhouse

	Integrate from //UE4/Main/...@3812936 to //UE4/Dev-Console/...

Change 3726554 by Luke.Thatcher

	[CONSOLE] [+] Improved Scoped Named Events
	 - Added SCOPED_NAMED_EVENT(_FSTRING/_TEXT/_F) macros to allow use of string literals, FString and printf in scoped named event strings.
	 - Replaced explicit use of FPlatformMisc::Begin/EndNamedEvent throughout the engine with macros, so the events can be compiled out.
	 - Fixed performance issues on various platforms with named events. SCOPED_NAMED_EVENT macros were not considering whether the platform uses wide or ansi char strings.

Change 3813513 by Ben.Woodhouse

	CSV profiler refactor+ bug fix
	- Move the CSV profiler to core, so we can use it in modules other than engine
	- Profiler no longer enqueues RT commands - this is handled by a new core delegate type
	- Make begin/end requests more robust, enqueued via a command queue and processed in order
	- Defer end capture requests by a frame. This ensures stats on threads other than the gamethread (e.g. renderthread etc) are complete, so the last frame is not truncated
	- Fix long-standing bug with multiple captures where old/stale frames would appear in the first <128 frames of captures
	- Move unit stats and dynamic resolution out of the profiler itself and into the engine. Only frametime is recorded in the profiler now
	- Fix longstanding bug in first frame renderthread time in the engine

Change 3814039 by Ben.Woodhouse

	More Csvprofiler improvements
	- Event support (via CSV_EVENT macros). These appear in the "EVENTS" column of the CSV and can be used to add context to the stat data
	- Reduced memory overhead for timestamps through bit-packing (now 16 bytes per marker instead of 24), and splitting stats into different types

Change 3814229 by Ben.Woodhouse

	Integrate + refactor of CL 3792591 to reduce complexity and fix bugs

	Original changelist description:
	CSV profiler improvements:
	- The CSV profiler is now always compiled in on the server
	- The CSV profiler can now handle both int32 and float stats
	- In BeginCapture, the function can take additional arguments for some customization of filenames

	Fixes to the above:
	 - Remove FCustomValue class
	    - Fixes bug where all timestamp values from CSV_SCOPED_STATs were garbage (due to issues resulting from FCustomValue type ambiguity when adding values together)
	    - FCsvCustomStat now just uses simple union + bitfield flag to reduce size and complexity (4 bytes instead of 8 per value)
	    - FCsvColumn class modified to use doubles, which can represent both ints and floats without loss of precision - this class is not memory or time critical
	 - Replace multiple overrides used by the server for filenameprefix/folder/postfix etc with a single FilenameOverride

Change 3817339 by Ben.Woodhouse

	Duplicate from 3816641: CSV profiler improvements
	- Added a low-pri processing thread to compress raw timing data into a much more efficient format
	   - Reduces memory usage for 15 minute 30k frame capture with GPU stats enabled from 110MB to 3.4MB
	   - Processing time : 0.1ms per frame
	- Improved name handling for char literal stats. We now use the string as the ID rather than the pointer so multiple stat uses of the same name string apply only to  a single stat
	- Reworked thread data access to eliminate locking for anything time-critical. Frame boundaries now handled via a lock-free helper class
	-  Fixed bug with queue implementation where 1 in 128 reads would duplicate the last block of stat data
	-  Reduced #include dependencies for CSVProfiler.h
	-  Removed AccumulateMax because it doesn't work, and implementing properly would add a lot of complexity
	-  Added a simple test harness

Change 3827787 by Ben.Woodhouse

	Integrate-as-edit CL 3820678 from Fortnite/Main
	Allow the CSV Profiler to be compiled in to shipping dedicated server builds

Change 3827842 by Ben.Woodhouse

	Integrate-as-edit CL 3827079 from Fortnite/Main
	CSV profiler category support

Change 3840185 by Ben.Woodhouse

	Integrate from //UE4/Release-4.19/...@3840005 to //UE4/Dev-Console/...

Change 3861618 by Ben.Woodhouse

	Integrate from //UE4/Release-4.19/...@3860990 to //UE4/Dev-Console/...

Change 3901590 by Anthony.Bills

	Fix incorrect srv and sampler count when an array of textures or an array of samplers is used.

	- The code originally set the local variable BindCount but did not pass this to the total sampler or srv count and instead used the original BindDesc.BindCount (which will be 1) and still skipped over those resource indicies.
	- The code also checked for the array's bracket location but then only compared the resource names before the bracket, meaning non arrays could be included.

Change 3904065 by Anthony.Bills

	Allow easily increasing the maximum number of SRVs to 64 in the D3D12RHI.

	- Increasing to 64 will fix shader compilation issues when using more than 32 textures.
	- Casting has been added to correct the type on bitshifting the masks when the type changes.
	- A new type has been added for the bound VB mask to make sure bitshifting is explicitly typed.

Change 3905752 by Ben.Woodhouse

	Integrate from //UE4/Release-4.19/...@3904082 to //UE4/Dev-Console/...

Change 3920080 by Luke.Thatcher

	[CONSOLE] [+] Added .natvis visualizer for FRHICommandList, which expands the list links to a flat list of commands. To use this, add an FRHICommandList* to your watch window.

Change 3951698 by Keith.Judge

	Fix CIS (leftover debugging code)

	#jira UE-55443

Change 3953261 by Luke.Thatcher

	[CONSOLE] [^] Merging //UE4/Release-4.19 (CL 3944462) to Dev-Console (//UE4/Dev-Console)

Change 3971144 by Luke.Thatcher

	[CONSOLE] [^] Merging //UE4/Dev-Main (CL 3964322) to Dev-Console (//UE4/Dev-Console)
	 - First merge since Dev-Console was reparented back to Dev-Main.

Change 3992843 by Anthony.Bills

	Fix deploying to the default device in UAT.

Change 4010029 by Marcin.Undak

	ShooterGame: use console UI on Quail platform

Change 4010030 by Marcin.Undak

	RecordPerformance: don't fail if game crashes

Change 4012181 by Marcin.Undak

	Implemented Low Level Memory tracker for Unix platforms

Change 4039094 by Ben.Woodhouse

	[INTEGRATE] Integrate from //UE4/Main/...@4039044 to //UE4/Dev-Console/...

Change 4039598 by David.Harvey

	adding LLM hooks for bespoke platform-specific memory tracking tools.

Change 4042637 by Arciel.Rekman

	Merging //UE4/Dev-Main@4042226 to Dev-Console (//UE4/Dev-Console)

Change 4044679 by Marcin.Undak

	RecordPerformance: store logs from runs

Change 4045895 by Marcin.Undak

	Vulkan: added log when compiling new PSO

Change 4047096 by Ian.Fox

	#ShooterGame - Potential fix for ShooterGame friendslist empty issue

	#review-4046664

Change 4048332 by Marcin.Undak

	Compilation fix for Oculus. Ported Rolando's change from Dev-Rendering

Change 4051609 by Marcin.Undak

	Compilation fix

Change 4053542 by Joe.Barnes

	Integrate as edit:

	Faster Guid creation. Don't check time for every Guid (it's slow on some platforms). Check it once, then use that + CPU timer.

Change 4057462 by Luke.Thatcher

	[CONSOLE] [-] Remove "frame rate smoothing" override on console.
	 - This was arbitrarily forced on when vsync is enabled.
	 - It can still be enabled via the bSmoothFrameRate in UEngine if needed.

	(merge as edit CL 3891390 from //Fortnite/Release-3.0.0 to //UE4/Dev-Console)

Change 4057551 by Luke.Thatcher

	[CONSOLE] [~] Improvements to game thread hitch detection.
	 - Allows the system to be configured by ini files.
	 - Fixes callstack addresses being truncated to 32 bits.
	 - Removes memory allocation in hitch callstack logging code. We use a pre-allocated 64KB buffer for this now. This was a memory leak anyway, as the memory was never freed.
	 - Performs a throwaway gamethread stack dump on launch, to prime the stack walker so we don't make gameplay hitches worse than they otherwise would be.

	#jira FORT-78417

	(merge as edit CL 3967234 from //Fortnite/Main to //UE4/Dev-Console)

Change 4057562 by Luke.Thatcher

	[CONSOLE] [~] Increase verbosity on FlushAsyncLoading logging. Higher verbosity limited to once per frame, to avoid log spam.

	#jira FORT-77411

	(merge as edit CL 3967336 from //Fortnite/Release-3.4 to //UE4/Dev-Console)

Change 4057568 by Luke.Thatcher

	[CONSOLE] [+] Add async load flush analytics stats.

	#jira FORT-80229

	(merge as edit CL 3986900 from //Fortnite/Release-3.5 to //UE4/Dev-Console)

Change 4057590 by Luke.Thatcher

	[CONSOLE] [+] Add hang detector duration multiplier to allow extending the hang timeout on loading screens etc.

	#jira FORT-80325

	(merge as edit CL 3990275 from //Fortnite/Release-3.5 to //UE4/Dev-Console)

Change 4057679 by Luke.Thatcher

	[CONSOLE] [WIN64] [~] Modify hitch detector stack walking to avoid expensive symbol lookups. Stack walk now only prints the return addresses without symbol information.
	 - Old behaviour of performing full symbol resolution can be achieved by defining LOOKUP_SYMBOLS_IN_HITCH_STACK_WALK to 1.
	 - Also fixed a minor bug in CaptureStackTraceHelper which duplicates the last line if stack walking fails.

	#jira FORT-87620

	(merge as edit CL 4049361 from //Fortnite/Release-4.1 to //UE4/Dev-Console)

Change 4057779 by Luke.Thatcher

	[CONSOLE] [~] Refactored how the default LLM categories are defined.
	 - Prevents initialization order problems which stops early boot code from getting group name strings.

	#jira FORT-80324

	(partial merge as edit CL 4057155 from //Fortnite/Main to //UE4/Dev-Console)

Change 4059809 by Luke.Thatcher

	[CONSOLE] [!] Fix compiler warning (incorrect order of member initializers in FGameThreadHitchHeartBeat)

Change 4060781 by Luke.Thatcher

	[CONSOLE] [!] Fix for XGE control worker hanging around after killing the editor from Visual Studio.
	 - Reused the "XGE monitor" mode of shader compile worker, that the old XGE compiler used.
	 - The monitor watches the editor process and automatically terminates the build if the editor exits.

	#jira UE-55864

Change 4063641 by David.Harvey

	remove legacy "r.AmbientOcclusionSampleSetQuality" cvar from ini files (deleted in CL 2622264)... gets rid of warning in log.

	#jira UE-51256

Change 4067367 by Ben.Woodhouse

	Edigrate dev rendering CL 4062236:
	AppleTV doesn't appear to support the set*Bytes APIs in Metal.

	#jira UE-58580

Change 4067748 by Arciel.Rekman

	Fix editor build (a plugin also needs private VulkanRHI headers, heh).

Change 4068121 by Arciel.Rekman

	Correct paths to referenced directories for Linux.

	- Should help CIS.

Change 4069919 by Arciel.Rekman

	Fix Lumin build (another plugin needs private Vulkan headers, grrr).

[CL 4070552 by Luke Thatcher in Main branch]
2018-05-14 09:49:35 -04:00

101 lines
3.7 KiB
C#

// Copyright 1998-2018 Epic Games, Inc. All Rights Reserved.
using System;
using System.IO;
using System.Text;
using System.Threading;
using Tools.DotNETCommon;
using UnrealBuildTool;
namespace AutomationTool
{
/// <summary>
/// Utility class which creates a file and obtains an exclusive lock on it. Used as a mutex between processes on different machines through a network share.
/// </summary>
static class LockFile
{
private static void TakeLock(DirectoryReference LockDirectory, TimeSpan Timeout, System.Action Callback)
{
string LockFilePath = Path.Combine(LockDirectory.FullName, ".lock");
FileStream Stream = null;
DateTime StartTime = DateTime.Now;
DateTime Deadline = StartTime.Add(Timeout);
try
{
DirectoryReference.CreateDirectory(LockDirectory);
for (int Iterations = 0; ; ++Iterations)
{
// Attempt to create the lock file. Ignore any IO exceptions. Stream will be null if this fails.
try { Stream = new FileStream(LockFilePath, FileMode.Create, FileAccess.Write, FileShare.Read, 4096, FileOptions.DeleteOnClose); }
catch (IOException) { }
if (Stream != null)
{
// If we have a stream, we've taken the lock.
try
{
// Write the machine name to the file.
Stream.Write(Encoding.UTF8.GetBytes(Environment.MachineName));
Stream.Flush();
break;
}
catch
{
throw new AutomationException("Failed to write to the lock file '{0}'.", LockFilePath);
}
}
// We've failed to take the lock. Throw an exception if the timeout has elapsed.
// Otherwise print a log message and retry.
var CurrentTime = DateTime.Now;
if (CurrentTime >= Deadline)
{
throw new AutomationException("Couldn't create lock file '{0}' after {1} seconds.", LockFilePath, CurrentTime.Subtract(StartTime).TotalSeconds);
}
if (Iterations == 0)
{
CommandUtils.Log("Waiting for lock file '{0}' to be removed...", LockFilePath);
}
else if ((Iterations % 30) == 0)
{
CommandUtils.Log("Still waiting for lock file '{0}' after {1} seconds.", LockFilePath, CurrentTime.Subtract(StartTime).TotalSeconds);
}
// Wait for a while before retrying.
Thread.Sleep(1000);
}
// Invoke the user callback now that we own the lock.
Callback();
}
finally
{
// Always dispose the lock file stream if we took the lock.
// The file will delete on close.
if (Stream != null)
{
Stream.Dispose();
Stream = null;
}
}
}
public static void OptionallyTakeLock(bool Condition, DirectoryReference LockDirectory, TimeSpan Timeout, System.Action Callback)
{
if (Condition)
{
TakeLock(LockDirectory, Timeout, Callback);
}
else
{
// No lock required, invoke the callback directly.
Callback();
}
}
}
}