Files
UnrealEngineUWP/Engine/Source/Programs/Mac/ShaderCacheTool/Private/ShaderCacheTool.cpp

106 lines
2.3 KiB
C++
Raw Normal View History

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.
Copying //UE4/Dev-Platform to //UE4/Main ========================== MAJOR FEATURES + CHANGES ========================== Change 2719147 on 2015/10/07 by Mark.Satterthwaite Allow the shader cache to perform some precompilation synchronously on load before falling back to asynchronous compilation to balance load times against total time spent precompiling. Added a stat to the group that reports how long the precompile has been running until it completes so it is easier to track. Change 2719182 on 2015/10/07 by Mark.Satterthwaite Refactor the ShaderCache's internal data structures and change the way we handle recording whether a particular predraw state has been submitted to try and make it more efficient. Change 2719185 on 2015/10/07 by Mark.Satterthwaite Merging CL #2717701: Try and fix random crashes on Mac when manipulating bound-shader-states caused by ShaderCache potentially providing a bogus shader state pointer on exit from predraw. Change 2719434 on 2015/10/07 by Mark.Satterthwaite Make sure that Mac ensures reports have a source context and a sane callstack when sent to the crash-reports server. Change 2724764 on 2015/10/12 by Josh.Adams [Initial AppleTV support] Merging //depot/YakBranch/... to //UE4/Dev-Platform/... Change 2726266 on 2015/10/13 by Lee.Clark PS4 - Calc reserve size required for DMA copy when using unsafe command buffers Change 2726401 on 2015/10/13 by Mark.Satterthwaite Merging CL #2716418: Fix UE-15228 'Crash Report Client doesn't restart into project editor on Mac' by reporting the original command line supplied by LaunchMac, not the modified one that strips the project name. The CRC can then relaunch as expected. #jira UE-15228 Change 2726421 on 2015/10/13 by Lee.Clark PS4 - Don't try to clear invalid targets Change 2727040 on 2015/10/13 by Michael.Trepka Merging CL 2724777 - Fixed splash screen rendering for images with DPI different than 72 Change 2729783 on 2015/10/15 by Keith.Judge Fix huge memory leak in Test/Shipping configurations, caused because I am a numpty. Change 2729847 on 2015/10/15 by Mark.Satterthwaite Merging CL #2729846: On OS X unconstrain windows from the dimension of the parent display when in Windowed mode - it is OK for them to be larger in this case. They do need to be repositioned if on the Primary display so that they don't creep under the menu bar and become unmovable/unclosable and Fullscreen windows still need to be constrained to a single display. We can now take screenshots of windows that are larger than the display & not get grey bars beyond the cutoff. #jira UE-21992 Change 2729865 on 2015/10/15 by Keith.Judge Fast semantics - Finish up resource transitions, adding resource decompression where appropriate and using non-fast clears where we can't determine the resource transition. Change 2729897 on 2015/10/15 by Keith.Judge Fast Semantics - Make sure all GetData() calls are made safe with GPU fences. Change 2729972 on 2015/10/15 by Keith.Judge Removed the last vestiges of ID3D11DeviceContext/ID3D11DeviceContext1 from the Xbox RHI. Everything now uses ID3D11DeviceContextX directly. This should be marginally quicker as it stops a double call to ClearState(). Change 2731503 on 2015/10/16 by Keith.Judge Added _XDK_VERSION to the DDC key for textures, which should solve the issue of the tiling mode changing in August XDK (and future changes Microsoft may inflict). Change 2731596 on 2015/10/16 by Keith.Judge Fast Semantics - Add deferred resource deletion queue to make deleted resources be actually deleted a number of frames later so that the GPU is definitely finished with them. Hooked up the temporary SRVs for dynamic VBs as a first step. Change 2731928 on 2015/10/16 by Michael.Trepka PR #1659: Mac/Build.sh handles additional arguments (Contributed by judgeaxl) Change 2731934 on 2015/10/16 by Michael.Trepka PR #1618: added clang 3.7.0 -Wshift-negative-value ignore in JpegImageWrapper.cpp (Contributed by bsekura) Change 2732018 on 2015/10/16 by Mark.Satterthwaite Emit a shader code cache for each platforms requested shader formats, this is separate to the targeted formats as not all can or need to be cached. - The implementation extends the ShaderCache's hooks in FShaderResource's serialisation function to capture the required shaders. - Each target platform has its own list of cached shader formats, analogous to the list of targeted RHIs. Presently only the Mac implements this. - Code cached shaders are now compressed (for size) to reduce the overhead associated with keeping all the shader code around - this works esp. well for text-based formats like GLSL. Change 2732365 on 2015/10/16 by Josh.Adams - Packaging a TVOS .ipa now works (still haven't tried any of the Editor integration like Launch On) Change 2733170 on 2015/10/18 by Terence.Burns Fix for Android IAP query not returning entire inventory. Change 2733174 on 2015/10/18 by Terence.Burns Fix Movie player issue where wait for movie to finish isnt being respected. Seems a stray bUserCanceled event flag was causing this not to be observed. Added some verbose logging to apple movie player. Change 2733488 on 2015/10/19 by Mark.Satterthwaite Added the ability to merge the .ushadercache files used by the ShaderCache to store shader & draw state information. - Fixed a bug that would cause invalid shader membership and draw state information to be logged. - Added a separate command-line tool to merge shader cache files, currently Mac-only but in theory should work on other platforms too. Change 2735226 on 2015/10/20 by Mark.Satterthwaite Fix temporal AA rendering on GL/Mac OS X - you can't rely on EyeAdaptation values unless SM5 is available so only perform that code on SM5 & we must correctly clamp saturate(NaN) to 0 as the current hlslcc won't do that for us (& is required by the HLSL spec). The latter used to be clamped in the AA_ALPHA && AA_VELOCITY_WEIGHTING code block that was removed recently. #jira UE-21214 #jira UE-19913 Change 2736722 on 2015/10/21 by Daniel.Lamb Improved performance of cooking stats system. Change 2737172 on 2015/10/21 by Daniel.Lamb Improved cooking stats performance for ddc stats.
2015-12-10 16:56:55 -05:00
// ShaderCacheTool.cpp: Driver for testing compilation of an individual shader
#include "ShaderCacheTool.h"
#include "ShaderCache.h"
#include "RequiredProgramMainCPPInclude.h"
DEFINE_LOG_CATEGORY(LogShaderCacheTool);
IMPLEMENT_APPLICATION(ShaderCacheTool, "ShaderCacheTool");
namespace SCT
{
FRunInfo::FRunInfo() :
InputFile1(""),
InputFile2(""),
OutputFile("ShaderDrawCache.ushadercache"),
GameVersion(0)
{
}
void PrintUsage()
{
UE_LOG(LogShaderCacheTool, Display, TEXT("Usage:"));
UE_LOG(LogShaderCacheTool, Display, TEXT("\tShaderCacheTool[.exe] FilePath1 FilePath2 [OutputPath] [-version=%d]"));
UE_LOG(LogShaderCacheTool, Display, TEXT("\t\tOptions:"));
UE_LOG(LogShaderCacheTool, Display, TEXT("\t\t\t-o=version\tThe game version to set."));
}
bool FRunInfo::Setup(const TArray<FString>& InOptions, const TArray<FString>& InSwitches)
{
if(InOptions.Num() >= 2)
{
InputFile1 = InOptions[0];
InputFile2 = InOptions[1];
if(InOptions.Num() >= 3)
{
OutputFile = InOptions[2];
}
for(FString Switch : InSwitches)
{
if(Switch.StartsWith(TEXT("version=")))
{
FString Vers = Switch.Mid(8);
if(Vers.IsNumeric())
{
Copying //UE4/Dev-Core to //UE4/Dev-Main (Source: //UE4/Dev-Core @ 3228803) #lockdown Nick.Penwarden #rb none ========================== MAJOR FEATURES + CHANGES ========================== Change 3209807 on 2016/11/24 by Chris.Wood CRP now has improved handling of lost comms with website/DB (CRP v.1.2.9) [UE-38397] - CrashReportProcess should not lose crashes when the website goes down. Change 3209936 on 2016/11/24 by Steven.Hutton Reconciled offline work. Fixed a number of display bugs with the reports page. Change 3209938 on 2016/11/24 by Steven.Hutton Reconciled offline work Adding JQuery UI packages Change 3210736 on 2016/11/28 by Steve.Robb Unset() made protected, which is an implementation details of TFunction and not the way to unbind one (may leak memory). Fixes to existing calls. https://answers.unrealengine.com/questions/494947/proper-way-to-deal-with-destroying-a-tfunction.html Change 3211181 on 2016/11/28 by Steve.Robb Improved error message when binding a delegate to a pending kill object. #jira UE-5232 Change 3211214 on 2016/11/28 by Steve.Robb PR #2978: fixed typo in FMallocLeakDetection (Contributed by finap) #jira UE-39049 Change 3211301 on 2016/11/28 by Steve.Robb PR #2892: Copy bCustomVersionsAreReset when creating an Archive from another Ar. (Contributed by surakin) #jira UE-37941 Change 3213387 on 2016/11/29 by Steven.Hutton Test of a release note parsed from the changelist Change 3213553 on 2016/11/29 by Gil.Gribb UE4 - Rework dependencies and UStruct creation so that we can create UBlueprintGeneratedClass's without needing the parent class serialized. Change 3214800 on 2016/11/30 by Robert.Manuszewski ModuleManager will now use a critical section instead of FMultiReaderSingleWriterGT to make it really thread safe. - removed FMultiReaderSingleWriterGT because it was not fully thread safe Change 3214926 on 2016/11/30 by Robert.Manuszewski Merging using Dev-Core_To_Dev-LoadTimes (reversed) Change 3214981 on 2016/11/30 by Gil.Gribb UE4 - Make sure that subobjects of CDOs are exported even if they don't have any direct references; they might be archetypes. Change 3215392 on 2016/11/30 by Steve.Robb Error out on editor builds when reading a non-boolean value in an archive. Change 3215674 on 2016/11/30 by Steve.Robb Replacement of a custom scope-exit setup with our standard macro. Change 3215720 on 2016/11/30 by Steve.Robb Default constructor for TTuple which value-initializes the elements. Change 3216777 on 2016/12/01 by Graeme.Thornton Add NoRedist and NotForLicensees folders in the game folder to the C# version of config file hierarchy. Change 3216858 on 2016/12/01 by Graeme.Thornton Improvements to pak signing and encryption - Remove compile time defines for encryption key and signing keys - Embed keys in executable by including in the UBT generated UELinkerFixups.cpp file - Add key access core delegate for passing these keys through to pak platform file Change 3216860 on 2016/12/01 by Graeme.Thornton Re-enable pak signing and encryption in ShooterGame Change 3216861 on 2016/12/01 by Graeme.Thornton Re-enable pak signing and encryption in Paragon Change 3217076 on 2016/12/01 by Gil.Gribb UE4 - replaced !GIsIntialLoad with EVENT_DRIVEN_ASYNC_LOAD_ACTIVE_AT_RUNTIME as a first step toward boot time EDL Change 3221139 on 2016/12/05 by Ben.Woodhouse Dummy integrate of CL 3221131 (versioning workarounds) from Dev-LoadTimes4.14 with accept target to prevent those workarounds being merged out of dev-loadtimes4.14. Command: p4 integrate //UE4/Dev-LoadTimes4.14/...@3221131,3221131 w/ resolve/accept-target Change 3221410 on 2016/12/05 by Steve.Robb Allow Algo::IsSorted to work on C arrays and initializer_lists. Change 3221673 on 2016/12/05 by Gil.Gribb set up an intentional merge conflict so we can do the right thing when dev-BP and dev-core come together Change 3223182 on 2016/12/06 by Chris.Wood Add EngineModeEx to CrashReportCommon [UE-39258] - Update CrashReporter to include whether Engine crashes are from vanilla build or not Change 3224646 on 2016/12/07 by Robert.Manuszewski It's no longer necessary to rebuild the cooked exe to enable the event driven loader. EDL and the new async IO are now controlled with ini settings only. Change 3224647 on 2016/12/07 by Robert.Manuszewski Event Driven Loader is now enabled by default. Change 3224669 on 2016/12/07 by Chris.Wood Compressing some crash files output from CRP to S3 (CRP v1.2.11) [UE-37564] - Add compression support for CRP crash files in S3 Add EngineModeEx to support to CRP (CRP v1.2.10) [UE-39258] - Update CrashReporter to include whether Engine crashes are from vanilla build or not Change 3224873 on 2016/12/07 by Robert.Manuszewski A few fixes to the EDL macro refactor. Change 3224933 on 2016/12/07 by Robert.Manuszewski Async Loading settings will now be logged on startup in cooked games Change 3224984 on 2016/12/07 by Robert.Manuszewski Small fix to EDL and new async IO global var init. Change 3225027 on 2016/12/07 by Robert.Manuszewski Re-enabling EDL in ShooterGame since it no longer requires the cooked exe to be re-compiled (and EDL is on by default anyway). Change 3226702 on 2016/12/08 by Steve.Robb UHT can now skip over macro-like identifiers while parsing. Correctly handles FTypefaceEntry::Name parsing, which was broken by the workaround in CL# 3219332. Syntax workaround reverted in FTypefaceEntry. #jira UE-38231 Change 3226756 on 2016/12/08 by Steve.Robb LexicalConversion renamed to Lex after #core discussion. Lex::ToString overloads added for FString itself. Change 3226766 on 2016/12/08 by Steve.Robb FCookStatsManager::ToString replaced with Lex::ToString, as it now supports the required functionality. Change 3226949 on 2016/12/08 by Robert.Manuszewski Fixing static analysis warnings Change 3228566 on 2016/12/09 by Robert.Manuszewski Making UBT not think it needs to use separate temporary target for non source code projects when EDL settings don't match default. Change 3228601 on 2016/12/09 by Steve.Robb Fix for TSparseArray visualization of types smaller than 8 bytes. Change 3228654 on 2016/12/09 by Gil.Gribb UE4 - Fix very old bug with lock free lists, probably not relevant to anything we currently use. Change 3228697 on 2016/12/09 by Graeme.Thornton Rebuilt UnrealPak [CL 3228932 by Robert Manuszewski in Main branch]
2016-12-09 11:36:14 -05:00
Lex::FromString(GameVersion, *Vers);
Copying //UE4/Dev-Platform to //UE4/Main ========================== MAJOR FEATURES + CHANGES ========================== Change 2719147 on 2015/10/07 by Mark.Satterthwaite Allow the shader cache to perform some precompilation synchronously on load before falling back to asynchronous compilation to balance load times against total time spent precompiling. Added a stat to the group that reports how long the precompile has been running until it completes so it is easier to track. Change 2719182 on 2015/10/07 by Mark.Satterthwaite Refactor the ShaderCache's internal data structures and change the way we handle recording whether a particular predraw state has been submitted to try and make it more efficient. Change 2719185 on 2015/10/07 by Mark.Satterthwaite Merging CL #2717701: Try and fix random crashes on Mac when manipulating bound-shader-states caused by ShaderCache potentially providing a bogus shader state pointer on exit from predraw. Change 2719434 on 2015/10/07 by Mark.Satterthwaite Make sure that Mac ensures reports have a source context and a sane callstack when sent to the crash-reports server. Change 2724764 on 2015/10/12 by Josh.Adams [Initial AppleTV support] Merging //depot/YakBranch/... to //UE4/Dev-Platform/... Change 2726266 on 2015/10/13 by Lee.Clark PS4 - Calc reserve size required for DMA copy when using unsafe command buffers Change 2726401 on 2015/10/13 by Mark.Satterthwaite Merging CL #2716418: Fix UE-15228 'Crash Report Client doesn't restart into project editor on Mac' by reporting the original command line supplied by LaunchMac, not the modified one that strips the project name. The CRC can then relaunch as expected. #jira UE-15228 Change 2726421 on 2015/10/13 by Lee.Clark PS4 - Don't try to clear invalid targets Change 2727040 on 2015/10/13 by Michael.Trepka Merging CL 2724777 - Fixed splash screen rendering for images with DPI different than 72 Change 2729783 on 2015/10/15 by Keith.Judge Fix huge memory leak in Test/Shipping configurations, caused because I am a numpty. Change 2729847 on 2015/10/15 by Mark.Satterthwaite Merging CL #2729846: On OS X unconstrain windows from the dimension of the parent display when in Windowed mode - it is OK for them to be larger in this case. They do need to be repositioned if on the Primary display so that they don't creep under the menu bar and become unmovable/unclosable and Fullscreen windows still need to be constrained to a single display. We can now take screenshots of windows that are larger than the display & not get grey bars beyond the cutoff. #jira UE-21992 Change 2729865 on 2015/10/15 by Keith.Judge Fast semantics - Finish up resource transitions, adding resource decompression where appropriate and using non-fast clears where we can't determine the resource transition. Change 2729897 on 2015/10/15 by Keith.Judge Fast Semantics - Make sure all GetData() calls are made safe with GPU fences. Change 2729972 on 2015/10/15 by Keith.Judge Removed the last vestiges of ID3D11DeviceContext/ID3D11DeviceContext1 from the Xbox RHI. Everything now uses ID3D11DeviceContextX directly. This should be marginally quicker as it stops a double call to ClearState(). Change 2731503 on 2015/10/16 by Keith.Judge Added _XDK_VERSION to the DDC key for textures, which should solve the issue of the tiling mode changing in August XDK (and future changes Microsoft may inflict). Change 2731596 on 2015/10/16 by Keith.Judge Fast Semantics - Add deferred resource deletion queue to make deleted resources be actually deleted a number of frames later so that the GPU is definitely finished with them. Hooked up the temporary SRVs for dynamic VBs as a first step. Change 2731928 on 2015/10/16 by Michael.Trepka PR #1659: Mac/Build.sh handles additional arguments (Contributed by judgeaxl) Change 2731934 on 2015/10/16 by Michael.Trepka PR #1618: added clang 3.7.0 -Wshift-negative-value ignore in JpegImageWrapper.cpp (Contributed by bsekura) Change 2732018 on 2015/10/16 by Mark.Satterthwaite Emit a shader code cache for each platforms requested shader formats, this is separate to the targeted formats as not all can or need to be cached. - The implementation extends the ShaderCache's hooks in FShaderResource's serialisation function to capture the required shaders. - Each target platform has its own list of cached shader formats, analogous to the list of targeted RHIs. Presently only the Mac implements this. - Code cached shaders are now compressed (for size) to reduce the overhead associated with keeping all the shader code around - this works esp. well for text-based formats like GLSL. Change 2732365 on 2015/10/16 by Josh.Adams - Packaging a TVOS .ipa now works (still haven't tried any of the Editor integration like Launch On) Change 2733170 on 2015/10/18 by Terence.Burns Fix for Android IAP query not returning entire inventory. Change 2733174 on 2015/10/18 by Terence.Burns Fix Movie player issue where wait for movie to finish isnt being respected. Seems a stray bUserCanceled event flag was causing this not to be observed. Added some verbose logging to apple movie player. Change 2733488 on 2015/10/19 by Mark.Satterthwaite Added the ability to merge the .ushadercache files used by the ShaderCache to store shader & draw state information. - Fixed a bug that would cause invalid shader membership and draw state information to be logged. - Added a separate command-line tool to merge shader cache files, currently Mac-only but in theory should work on other platforms too. Change 2735226 on 2015/10/20 by Mark.Satterthwaite Fix temporal AA rendering on GL/Mac OS X - you can't rely on EyeAdaptation values unless SM5 is available so only perform that code on SM5 & we must correctly clamp saturate(NaN) to 0 as the current hlslcc won't do that for us (& is required by the HLSL spec). The latter used to be clamped in the AA_ALPHA && AA_VELOCITY_WEIGHTING code block that was removed recently. #jira UE-21214 #jira UE-19913 Change 2736722 on 2015/10/21 by Daniel.Lamb Improved performance of cooking stats system. Change 2737172 on 2015/10/21 by Daniel.Lamb Improved cooking stats performance for ddc stats.
2015-12-10 16:56:55 -05:00
}
}
else
{
return false;
}
}
return true;
}
return false;
}
static int32 Run(const FRunInfo& RunInfo)
{
FShaderCache::SetGameVersion(RunInfo.GameVersion);
bool bOK = FShaderCache::MergeShaderCacheFiles(RunInfo.InputFile1, RunInfo.InputFile2, RunInfo.OutputFile);
return ((int32)!bOK);
}
}
INT32_MAIN_INT32_ARGC_TCHAR_ARGV()
{
GEngineLoop.PreInit(ArgC, ArgV, TEXT("-NOPACKAGECACHE -Multiprocess"));
TArray<FString> Tokens, Switches;
FCommandLine::Parse(FCommandLine::Get(), Tokens, Switches);
if (Tokens.Num() < 2)
{
UE_LOG(LogShaderCacheTool, Error, TEXT("Missing input files!"));
SCT::PrintUsage();
return 1;
}
if (Tokens.Num() == 2)
{
UE_LOG(LogShaderCacheTool, Warning,TEXT("Output set to ./ShaderDrawCache.ushadercache!"));
}
if (Tokens.Num() > 3)
{
UE_LOG(LogShaderCacheTool, Warning,TEXT("Ignoring extra command line arguments!"));
}
SCT::FRunInfo RunInfo;
if (!RunInfo.Setup(Tokens, Switches))
{
SCT::PrintUsage();
return 1;
}
int32 Value = SCT::Run(RunInfo);
return Value;
}