izzy/UnrealEngineUWP
0
0
Fork 0
mirror of https://github.com/izzy2lost/UnrealEngineUWP.git synced 2026-03-26 18:15:20 -07:00
Code Issues Packages Projects Releases Wiki Activity
Files
5.5
UnrealEngineUWP/Engine/Source/Runtime/RenderCore/Private/ShaderSource.cpp

111 lines
3.4 KiB
C++
Raw Permalink Normal View History

[shaders] - add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang. - add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types - add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding) #rb Jason.Nadro, Yuriy.ODonnell [CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
// Copyright Epic Games, Inc. All Rights Reserved.
#include "ShaderSource.h"
#include "Containers/StringConv.h"
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
#include "Compression/OodleDataCompression.h"
#include "HAL/IConsoleManager.h"
[shaders] - add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang. - add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types - add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding) #rb Jason.Nadro, Yuriy.ODonnell [CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
#include "HAL/UnrealMemory.h"
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
static int32 GShaderSourceCompressionMethod = 2;
static FAutoConsoleVariableRef CVarShaderSourceCompressionMethod(
TEXT("r.ShaderSource.CompressionMethod"),
GShaderSourceCompressionMethod,
TEXT("Compression method for shader source stored in memory. See FOodleDataCompression::ECompressor enum for supported values; defaults to Mermaid."),
ECVF_Default);
static int32 GShaderSourceCompressionLevel = 1;
FAutoConsoleVariableRef CVarShaderSourceCompressionLevel(
TEXT("r.ShaderSource.CompressionLevel"),
GShaderSourceCompressionLevel,
TEXT("Compression level for shader source stored in memory. See FOodleDataCompression::ECompressionLevel enum for supported values; default is SuperFast."),
ECVF_Default);
[shaders] - add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang. - add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types - add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding) #rb Jason.Nadro, Yuriy.ODonnell [CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
FShaderSource::FShaderSource(FShaderSource::FViewType InSrc, int32 AdditionalSlack)
{
Set(InSrc, AdditionalSlack);
}
void FShaderSource::Set(FShaderSource::FViewType InSrc, int32 AdditionalSlack)
{
SetLen(InSrc.Len() + AdditionalSlack);
FShaderSource::CharType* SourceData = Source.GetData();
FMemory::Memcpy(SourceData, InSrc.GetData(), sizeof(FShaderSource::CharType) * InSrc.Len());
}
[shaders] add ANSICHAR path for FShaderSource and enable by default #rb Yuriy.ODonnell, Laura.Hermanns #jira UE-196110 [CL 30632304 by dan elksnitis in ue5-main branch]
2024-01-16 09:15:51 -05:00
#if !SHADER_SOURCE_ANSI
[shaders] - add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang. - add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types - add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding) #rb Jason.Nadro, Yuriy.ODonnell [CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
void FShaderSource::Set(FAnsiStringView InSrc)
{
TStringConvert<ANSICHAR, TCHAR> Convert;
SetLen(InSrc.Len());
Convert.Convert(Source.GetData(), Source.Num(), InSrc.GetData(), InSrc.Len());
}
[shaders] add ANSICHAR path for FShaderSource and enable by default #rb Yuriy.ODonnell, Laura.Hermanns #jira UE-196110 [CL 30632304 by dan elksnitis in ue5-main branch]
2024-01-16 09:15:51 -05:00
#endif
[shaders] - add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang. - add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types - add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding) #rb Jason.Nadro, Yuriy.ODonnell [CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
FShaderSource& FShaderSource::operator=(FShaderSource::FStringType&& InSrc)
{
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
SourceCompressed.Empty();
[shaders] - modify the input hash debug dump mechanism to output an empty "debughash_<hash>" file instead of a txt file with the hash in contents, and always dump these files for the instance of the job that actually compiled - the existing cvar will now just make it so these files are also dumped for jobs which hit in DDC or the job cache; we don't do this by default so there's only a single match for the debug hash for any given shader normally and it is inside the folder containing the full debug info, including those artifacts which are only output as a side effect of the compile step - add the same hash as the first line in the stripped source code, so "debughash_<hash>" can be used as a search term in Everything to quickly find debug info associated with a shader (i.e. when looking at a capture in renderdoc or similar) note: this is a resubmit with fixes for mac/linux issues (avoiding printfs and using string builders instead; since wchar_t and TCHAR are not the same size on these platforms using %ls does not work), and a further fix for problems encountered with source compression when using wide chars in preprocessing instead of ansi chars. #rb Laura.Hermanns #jira UE-209753 [CL 32542773 by dan elksnitis in ue5-main branch]
2024-03-27 10:15:04 -04:00
DecompressedCharCount = 0;
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
[shaders] - add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang. - add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types - add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding) #rb Jason.Nadro, Yuriy.ODonnell [CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
int32 InInitialLen = InSrc.Len();
// input char array already has a null terminator appended, so can add one less padding char
TArray<CharType>& InSrcArray = InSrc.GetCharArray();
InSrcArray.AddZeroed(ShaderSourceSimdPadding - 1);
Source = MoveTemp(InSrcArray);
return *this;
}
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
void FShaderSource::Compress()
{
FOodleDataCompression::ECompressor Compressor = static_cast<FOodleDataCompression::ECompressor>(GShaderSourceCompressionMethod);
if (Compressor == FOodleDataCompression::ECompressor::NotSet)
{
return;
}
checkf(!IsCompressed(), TEXT("FShaderSource is already compressed"));
FOodleDataCompression::ECompressionLevel CompressionLevel = static_cast<FOodleDataCompression::ECompressionLevel>(GShaderSourceCompressionLevel);
[shaders] - modify the input hash debug dump mechanism to output an empty "debughash_<hash>" file instead of a txt file with the hash in contents, and always dump these files for the instance of the job that actually compiled - the existing cvar will now just make it so these files are also dumped for jobs which hit in DDC or the job cache; we don't do this by default so there's only a single match for the debug hash for any given shader normally and it is inside the folder containing the full debug info, including those artifacts which are only output as a side effect of the compile step - add the same hash as the first line in the stripped source code, so "debughash_<hash>" can be used as a search term in Everything to quickly find debug info associated with a shader (i.e. when looking at a capture in renderdoc or similar) note: this is a resubmit with fixes for mac/linux issues (avoiding printfs and using string builders instead; since wchar_t and TCHAR are not the same size on these platforms using %ls does not work), and a further fix for problems encountered with source compression when using wide chars in preprocessing instead of ansi chars. #rb Laura.Hermanns #jira UE-209753 [CL 32542773 by dan elksnitis in ue5-main branch]
2024-03-27 10:15:04 -04:00
DecompressedCharCount = Source.Num();
int32 CompressionBufferSize = FOodleDataCompression::CompressedBufferSizeNeeded(GetDecompressedSize());
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
SourceCompressed.SetNumUninitialized(CompressionBufferSize);
int32 CompressedSize = FOodleDataCompression::Compress(
SourceCompressed.GetData(),
CompressionBufferSize,
Source.GetData(),
[shaders] - modify the input hash debug dump mechanism to output an empty "debughash_<hash>" file instead of a txt file with the hash in contents, and always dump these files for the instance of the job that actually compiled - the existing cvar will now just make it so these files are also dumped for jobs which hit in DDC or the job cache; we don't do this by default so there's only a single match for the debug hash for any given shader normally and it is inside the folder containing the full debug info, including those artifacts which are only output as a side effect of the compile step - add the same hash as the first line in the stripped source code, so "debughash_<hash>" can be used as a search term in Everything to quickly find debug info associated with a shader (i.e. when looking at a capture in renderdoc or similar) note: this is a resubmit with fixes for mac/linux issues (avoiding printfs and using string builders instead; since wchar_t and TCHAR are not the same size on these platforms using %ls does not work), and a further fix for problems encountered with source compression when using wide chars in preprocessing instead of ansi chars. #rb Laura.Hermanns #jira UE-209753 [CL 32542773 by dan elksnitis in ue5-main branch]
2024-03-27 10:15:04 -04:00
GetDecompressedSize(),
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
Compressor,
CompressionLevel);
SourceCompressed.SetNum(CompressedSize, EAllowShrinking::Yes);
Source.Empty();
}
void FShaderSource::Decompress()
{
if (IsCompressed())
{
[shaders] - modify the input hash debug dump mechanism to output an empty "debughash_<hash>" file instead of a txt file with the hash in contents, and always dump these files for the instance of the job that actually compiled - the existing cvar will now just make it so these files are also dumped for jobs which hit in DDC or the job cache; we don't do this by default so there's only a single match for the debug hash for any given shader normally and it is inside the folder containing the full debug info, including those artifacts which are only output as a side effect of the compile step - add the same hash as the first line in the stripped source code, so "debughash_<hash>" can be used as a search term in Everything to quickly find debug info associated with a shader (i.e. when looking at a capture in renderdoc or similar) note: this is a resubmit with fixes for mac/linux issues (avoiding printfs and using string builders instead; since wchar_t and TCHAR are not the same size on these platforms using %ls does not work), and a further fix for problems encountered with source compression when using wide chars in preprocessing instead of ansi chars. #rb Laura.Hermanns #jira UE-209753 [CL 32542773 by dan elksnitis in ue5-main branch]
2024-03-27 10:15:04 -04:00
Source.SetNumUninitialized(DecompressedCharCount);
FOodleDataCompression::Decompress(Source.GetData(), GetDecompressedSize(), SourceCompressed.GetData(), SourceCompressed.Num());
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
SourceCompressed.Empty();
[shaders] - modify the input hash debug dump mechanism to output an empty "debughash_<hash>" file instead of a txt file with the hash in contents, and always dump these files for the instance of the job that actually compiled - the existing cvar will now just make it so these files are also dumped for jobs which hit in DDC or the job cache; we don't do this by default so there's only a single match for the debug hash for any given shader normally and it is inside the folder containing the full debug info, including those artifacts which are only output as a side effect of the compile step - add the same hash as the first line in the stripped source code, so "debughash_<hash>" can be used as a search term in Everything to quickly find debug info associated with a shader (i.e. when looking at a capture in renderdoc or similar) note: this is a resubmit with fixes for mac/linux issues (avoiding printfs and using string builders instead; since wchar_t and TCHAR are not the same size on these platforms using %ls does not work), and a further fix for problems encountered with source compression when using wide chars in preprocessing instead of ansi chars. #rb Laura.Hermanns #jira UE-209753 [CL 32542773 by dan elksnitis in ue5-main branch]
2024-03-27 10:15:04 -04:00
DecompressedCharCount = 0;
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
}
}
[shaders] - add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang. - add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types - add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding) #rb Jason.Nadro, Yuriy.ODonnell [CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
FArchive& operator<<(FArchive& Ar, FShaderSource& Src)
{
[shaders] - modify the input hash debug dump mechanism to output an empty "debughash_<hash>" file instead of a txt file with the hash in contents, and always dump these files for the instance of the job that actually compiled - the existing cvar will now just make it so these files are also dumped for jobs which hit in DDC or the job cache; we don't do this by default so there's only a single match for the debug hash for any given shader normally and it is inside the folder containing the full debug info, including those artifacts which are only output as a side effect of the compile step - add the same hash as the first line in the stripped source code, so "debughash_<hash>" can be used as a search term in Everything to quickly find debug info associated with a shader (i.e. when looking at a capture in renderdoc or similar) note: this is a resubmit with fixes for mac/linux issues (avoiding printfs and using string builders instead; since wchar_t and TCHAR are not the same size on these platforms using %ls does not work), and a further fix for problems encountered with source compression when using wide chars in preprocessing instead of ansi chars. #rb Laura.Hermanns #jira UE-209753 [CL 32542773 by dan elksnitis in ue5-main branch]
2024-03-27 10:15:04 -04:00
Ar << Src.DecompressedCharCount;
[shaders] resubmit: add compression for FShaderSource objects and enable by default (Mermaid SuperFast compression). This halves the high watermark for the ShaderCompiler LLM scope in my testing (QAGame cold cook), with minimal measurable impact to preprocessing time (<3% on average, but this may just be noise). #rb Jason.Nadro, Laura.Hermanns #jira UE-209037 [CL 32400079 by dan elksnitis in ue5-main branch]
2024-03-21 10:54:56 -04:00
if (Src.IsCompressed())
{
Ar << Src.SourceCompressed;
}
else
{
Ar << Src.Source;
}
[shaders] - add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang. - add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types - add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding) #rb Jason.Nadro, Yuriy.ODonnell [CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
return Ar;
}
Reference in New Issue Copy Permalink