Commit Graph

165 Commits

Author SHA1 Message Date
marc audy
399bcf9971 Disable PVS warning V758
Silence V570 false positives for bit field assignments
Silence various other PVS warnings
#rnx

[CL 29706746 by marc audy in ue5-main branch]
2023-11-14 00:29:43 -05:00
christopher waters
6f3f63b186 Bindless Conversion Changes
- Removing the need for COMPILER_DEFINE on GetResourceFromHeap and GetSamplerFromHeap
- FShaderParameterParser constructor now requires a FPlatformConfiguration object that each shader format can configure.
- FPlatformConfiguration::GenerateBindlessAccess allows each platform to generate the code for accessing a bindless resource or sampler.
- Added EShaderParameterParserConfigurationFlags to specify support for stable constant buffers and bindless.
- Moved EBindlessParameterMode into EShaderParameterParserConfigurationFlags::BindlessUsesArrays
- While moving things around, I was able to convert some TCHAR* uses to FStringView

#jira UE-166341
#rb mihnea.balta

[CL 29533376 by christopher waters in ue5-main branch]
2023-11-07 16:26:20 -05:00
jason hoerner
0e18ea7e81 Shader Preprocessor: Roll TEXT macro substitution into the C preprocessor step, via custom macro callback feature, making it effectively free. Saves 3% overall on preprocess, but also represents a simplification by reducing the number of unique preprocess passes.
#jira UE-198496
#rnx
#rb dan.elksnitis yuriy.odonnell jason.nadro

[CL 29100104 by jason hoerner in ue5-main branch]
2023-10-25 16:01:11 -04:00
jason hoerner
4bb44a53c7 Shader Preprocessor: Compact commented out uniform buffer declarations out of the source code in CleanupUniformBufferCode. Saves 9% of overall preprocess time by avoiding comment scanning in minifier parse step. Also took this opportunity to remove the unused original CleanupUniformBufferCode implementation -- the new version has been stable, so we don't need a CVar toggle any more.
#rnx
#rb yuriy.odonnell jason.nadro

[CL 29095286 by jason hoerner in ue5-main branch]
2023-10-25 13:58:44 -04:00
christopher waters
ab21e17d39 Bindless Resources
- Splitting "Bindless Resource" shader parameter types into "Bindless SRV" and "Bindless UAV" to make sure reflection/binding validation is correct.

#jira UE-166341
#rb jeannoe.morissette

[CL 28995862 by christopher waters in ue5-main branch]
2023-10-20 23:12:44 -04:00
yuriy odonnell
57397e6c36 Add CPU profiler scope to UE::ShaderCompilerCommon::RemoveDeadCode
#rb none

[CL 28744091 by yuriy odonnell in ue5-main branch]
2023-10-13 04:02:29 -04:00
jason hoerner
bca274f611 Shader Compiler: CleanupUniformBufferCodeNew minor fix to account for escaped quotes in text. Continue string parsing if character before quote is a backslash. Note that there are no escaped quotes in any existing .usf or .ush file at the moment, so I don't think the bug should have caused any issues.
#rnx
#rb dan.elksnitis jason.nadro

[CL 28494850 by jason hoerner in ue5-main branch]
2023-10-05 08:25:49 -04:00
jason hoerner
6f67025ccc Shader Compiler: Optimized version of CleanupUniformBufferCode, called during preprocessing. New version is 3.8x faster, producing a 19% overall speedup in ConditionalPreprocessShader. Processes the shader in a single pass, scanning for potential compound identifiers (symbol tokens connected by dots and optional whitespace), then checking if they match known uniform buffer members. A few optimizations improve the speed of checking if an identifier matches, including an early out based on identifier length and start character, followed by organizing uniform buffers and their members by length, requiring only a small subset of members to be tested. Parsing uniform buffer definitions was also optimized to avoid memory allocation.
Doesn't produce bit identical preprocessed output, as it compacts whitespace from compound identifiers before testing them, whereas the original implementation compacts whitespace after matching.  Compacting first makes testing more efficient and simpler, since it doesn't need to take into account whitespace.  For A/B validation, I did add temporary debug code that stored and reverted un-matched identifiers, mimicking the old behavior, and it was identical across the board.

#rnx
#rb dan.elksnitis chris.waters jason.nadro

[CL 28313553 by jason hoerner in ue5-main branch]
2023-09-28 07:08:47 -04:00
dan elksnitis
4515e3b5f4 [shaders] add debug dump option to output "detailed" shader source, including the output of the preprocessing step unmodified, the stripped version used for hashing, and whatever the final source passed to the compiler is (can differ from the stripped version if further modifications are applied as part of the compile step). this is only done if the preprocessed job cache is enabled; otherwise (and if this cvar is disabled) the single output will either be the compiler-step-modified source if it exists, or the output of the preprocessing step.
#rb Jason.Hoerner

[CL 28274493 by dan elksnitis in ue5-main branch]
2023-09-27 11:02:18 -04:00
christopher waters
7f66db7ffc Shader preprocessing optimizations:
- Removing system calls from UniformBuffer parsing and processing.
- When searching for uses of a UniformBuffer, start after the end of the UniformBuffer declaration.
- Simplifying the code around UniformBuffer reference cleanup
- Cuts time in CleanupUniformBufferCode by 50-75% based on how large the shader is.

#jira UE-195103
#rb dan.elksnitis, jason.hoerner

[CL 28172021 by christopher waters in ue5-main branch]
2023-09-23 05:43:20 -04:00
christopher waters
bb0a274b8c Reworking UniformBuffer code generation to not use an intermediate struct
- When generating Uniform Buffer HLSL code, a 'static const struct {} Name' was created which has been causing compiler issues off and on for years.
- When RemoveUniformBuffersFromSource removed said struct, it parsed the struct's structure to figure out how to translate UniformBuffer.Member into UniformBuffer_Member in code that accessed Uniform Buffers.
- Now, instead of a struct, a "UniformBuffer <Name>" declaration is created and filled with mappings from "UniformBuffer.Member" into the correct code.
- This allows ConstantBuffer objects to be used differently from cbuffer where the former generates "UniformBuffer.Member" and the latter generates "UniformBuffer_Member"
- This also allows resources to always be translated to "UniformBuffer_Member"
- Shader define 'PLATFORM_SUPPORTS_UB_STRUCT' is now 'PLATFORM_SUPPORTS_CONSTANTBUFFER_OBJECT' and will be set on shader formats that support ConstantBuffer<> objects.
- Removed lots of unused code from BindlessResources.ush

#jira UE-195103
#rb dan.elksnitis, jeannoe.morissette

[CL 28058373 by christopher waters in ue5-main branch]
2023-09-20 17:33:36 -04:00
carl lloyd
34d53bc691 Added validation to ensure UAV/SRV types and UB sizes match the ones generated by the shader compiler
#rb jeannoe.morissette
#jira UE-82353

[CL 27399050 by carl lloyd in ue5-main branch]
2023-08-25 22:19:10 -04:00
jeannoe morissette
b070af8d82 VulkanRHI: Add quotes around the entrypoint name in the command line for cases where multiple entrypoints are possible (like RT hit group)
#rb carl.lloyd
[FYI] aleksander.netzel
#rnx

[CL 27204165 by jeannoe morissette in ue5-main branch]
2023-08-18 11:39:37 -04:00
dan elksnitis
5f85b46590 [shaders] remove unnecessary passing of hlslcc flags as an option to debug info output. this was just used to pass a switch on the SCW directcompile cmdline which SCW no longer does anything with. also cleaned up some additional unnecessary code related to these flags in backends that no longer use hlslcc at all.
#rb Laura.Hermanns

[CL 26685354 by dan elksnitis in ue5-main branch]
2023-07-28 15:58:01 -04:00
charles derousiers
2365ee5308 Change D3D diagnostic buffer to output readable assert message (file/line/condition/message).
Use with check() / checkf()

#rb dan.elksnitis, christopher.waters
[FYI] yuriy.odonnell

[CL 26680005 by charles derousiers in ue5-main branch]
2023-07-28 13:55:18 -04:00
dan elksnitis
be2cf9c56e [shaders] debug output changes - add optional shader code binary output, fix redundant write of additional outputs
#rb Laura.Hermanns

[CL 26182686 by dan elksnitis in ue5-main branch]
2023-06-22 10:53:19 -04:00
laura hermanns
fcd0debcbd Fix crash on empty string of preprocessed shader source.
#rb Dan.Elksnitis

[CL 26024402 by laura hermanns in ue5-main branch]
2023-06-15 16:15:45 -04:00
dan elksnitis
d688f1fb4d [shaders] debug output improvements/fixes
- remove redundant options struct initialization in DumpDebugShaderData - this was resulting in losing some of the options set in the input parameter options struct
- add explicit "DumpExtendedDebugShaderData" which includes the additional debug output functionality only previously available if an IShaderFormat implementation inherited from FBaseShaderFormat
- add "AdditionalOutputs" functionality which can be used by backends to dump additional (text-based) files following the file naming rules as dictated by the options struct with less boilerplate code
- add a helper for constructing a debug filename from the debug output options struct (some backends require constructing such filenames for i.e. contents of batchfiles; the helper reduces boilerplate code in such cases)
- fix condition for old path of dumping OutputHash.txt to not redundantly dump if shader format supports independent preprocessing and preprocessed cache is enabled

#rb Massimo.Tristano

[CL 25892792 by dan elksnitis in ue5-main branch]
2023-06-09 10:27:24 -04:00
dan elksnitis
06395bee56 [shaders] add Hash field to FShaderCompilerInput so debug output of InputHash.txt works correctly whether or not the preprocessed cache is enabled, and the IShaderFormat API for debug output of independent-preprocessing-enabled backends is cleaner
#rb Jason.Nadro

[CL 25814977 by dan elksnitis in ue5-main branch]
2023-06-06 09:55:43 -04:00
dan elksnitis
2acbcb9533 [shaders]
- add new fields to FShaderCompilerOutput which should be populated with the shader source and entry point name if modified by a backend compile process (in a format implementing the independent preprocessing API). this is intended to supercede "OptionalFinalShaderSource" for such shader formats, as well as being used as the source for preprocessed debug dumps when enabled.
- modify the PCD3D shader format to use these fields as intended (and so fix the debug output for pipelines which remove unused interpolators to match the final modified source).

#rb Laura.Hermanns
#preflight 64777f850848b7126dda194c

[CL 25706448 by dan elksnitis in ue5-main branch]
2023-05-31 13:31:39 -04:00
jason hoerner
88d847a0ad GlobalBeginCompileShader optimizations. 3.2x improvement in performance of the function on Lyra cook:
* Early out when adding uniform buffer resource entries, by checking if uniform buffer had been processed, before processing individual entries.
* Changed resource entry list from a map to an array, saving the cost of map operations adding items to it.
* Resource entries for global uniform buffers cached once at startup.
* Resource entry member names stored in an external buffer, to avoid per-entry string memory allocation overhead.
* Miscellaneous smaller optimizations -- for example, using ByHash to avoid redundant hashing and FString construction.

#jira none
#rnx
#rb jason.nadro dan.elksnitis
#preflight 646cfdbc1134ffac7034af60

[CL 25596308 by jason hoerner in ue5-main branch]
2023-05-23 20:16:11 -04:00
ionut matasaru
5189818fb8 Fixed mismatch string formatting argument in shader compiler (for hlslccflags).
#rb dan.elksnitis
#preflight 64568240fd4b8f4e0d49d747

[CL 25563481 by ionut matasaru in ue5-main branch]
2023-05-22 10:03:04 -04:00
dan elksnitis
f645f93588 [shaders] remove legacy preprocessor option & mcpp library
#rb Jason.Nadro
#preflight 6467bf0c2c0a5da0dcd7aaf2
#fyi Yuriy.ODonnell

[CL 25549427 by dan elksnitis in ue5-main branch]
2023-05-19 14:50:25 -04:00
dan elksnitis
ce8ef1b37c [shaders] fix debug info output when preprocessed job cache is disabled
#preflight 64594f5b9f4768296300db62
#rb Jason.Nadro

[CL 25386339 by dan elksnitis in ue5-main branch]
2023-05-09 08:52:52 -04:00
Steve Robb
0f44db07d5 Replaced some container operator new usage.
#rb james.hopkin
#jira none
#preflight 6459670f28155a0f41565a34

[CL 25384456 by Steve Robb in ue5-main branch]
2023-05-09 05:17:49 -04:00