Commit Graph

314 Commits

Author SHA1 Message Date
jason hoerner
6f67025ccc Shader Compiler: Optimized version of CleanupUniformBufferCode, called during preprocessing. New version is 3.8x faster, producing a 19% overall speedup in ConditionalPreprocessShader. Processes the shader in a single pass, scanning for potential compound identifiers (symbol tokens connected by dots and optional whitespace), then checking if they match known uniform buffer members. A few optimizations improve the speed of checking if an identifier matches, including an early out based on identifier length and start character, followed by organizing uniform buffers and their members by length, requiring only a small subset of members to be tested. Parsing uniform buffer definitions was also optimized to avoid memory allocation.
Doesn't produce bit identical preprocessed output, as it compacts whitespace from compound identifiers before testing them, whereas the original implementation compacts whitespace after matching.  Compacting first makes testing more efficient and simpler, since it doesn't need to take into account whitespace.  For A/B validation, I did add temporary debug code that stored and reverted un-matched identifiers, mimicking the old behavior, and it was identical across the board.

#rnx
#rb dan.elksnitis chris.waters jason.nadro

[CL 28313553 by jason hoerner in ue5-main branch]
2023-09-28 07:08:47 -04:00
dan elksnitis
4515e3b5f4 [shaders] add debug dump option to output "detailed" shader source, including the output of the preprocessing step unmodified, the stripped version used for hashing, and whatever the final source passed to the compiler is (can differ from the stripped version if further modifications are applied as part of the compile step). this is only done if the preprocessed job cache is enabled; otherwise (and if this cvar is disabled) the single output will either be the compiler-step-modified source if it exists, or the output of the preprocessing step.
#rb Jason.Hoerner

[CL 28274493 by dan elksnitis in ue5-main branch]
2023-09-27 11:02:18 -04:00
christopher waters
7f66db7ffc Shader preprocessing optimizations:
- Removing system calls from UniformBuffer parsing and processing.
- When searching for uses of a UniformBuffer, start after the end of the UniformBuffer declaration.
- Simplifying the code around UniformBuffer reference cleanup
- Cuts time in CleanupUniformBufferCode by 50-75% based on how large the shader is.

#jira UE-195103
#rb dan.elksnitis, jason.hoerner

[CL 28172021 by christopher waters in ue5-main branch]
2023-09-23 05:43:20 -04:00
christopher waters
bb0a274b8c Reworking UniformBuffer code generation to not use an intermediate struct
- When generating Uniform Buffer HLSL code, a 'static const struct {} Name' was created which has been causing compiler issues off and on for years.
- When RemoveUniformBuffersFromSource removed said struct, it parsed the struct's structure to figure out how to translate UniformBuffer.Member into UniformBuffer_Member in code that accessed Uniform Buffers.
- Now, instead of a struct, a "UniformBuffer <Name>" declaration is created and filled with mappings from "UniformBuffer.Member" into the correct code.
- This allows ConstantBuffer objects to be used differently from cbuffer where the former generates "UniformBuffer.Member" and the latter generates "UniformBuffer_Member"
- This also allows resources to always be translated to "UniformBuffer_Member"
- Shader define 'PLATFORM_SUPPORTS_UB_STRUCT' is now 'PLATFORM_SUPPORTS_CONSTANTBUFFER_OBJECT' and will be set on shader formats that support ConstantBuffer<> objects.
- Removed lots of unused code from BindlessResources.ush

#jira UE-195103
#rb dan.elksnitis, jeannoe.morissette

[CL 28058373 by christopher waters in ue5-main branch]
2023-09-20 17:33:36 -04:00
tom holmes
0c53e701a0 Improve ShaderSymbols.info file generation in multicook scenarios
Dedupe info where multiple workers built same shader

#rb Ben.Woodhouse
#tests local runs, compare of farm built files attached in bug

[CL 27995633 by tom holmes in ue5-main branch]
2023-09-19 10:46:28 -04:00
laura hermanns
f9b3974ab9 Platform-specific DXC improvements:
- Avoid explicit FMA instructions in platform-specific GLSL to let backend-compiler decide when to use them.
- Fix code refelction when using SPIR-V directly with backend-compiler.
- Define preset of SPIR-V optimization passes to reduce nested expressions "preset(relax-nested-expr)".

#rb Serge.Bernier, Brian.White
[FYI] Graeme.Thornton, Jason.Nadro, Carl.Lloyd, Florin.Pascu, Michael.WAnderson
#jira UE-195236, UE-194869

[CL 27994871 by laura hermanns in ue5-main branch]
2023-09-19 10:25:17 -04:00
laura hermanns
d7fff598d5 Allow passing custom SPIV-V optimization passes to ShaderConductor. This allows us to control more fine-grained what optimization passes work best for certain render passes on different platforms.
#rb Serge.Bernier
[FYI] Carl.Lloyd, JeanNoe.Morissette
#rnx

[CL 27880727 by laura hermanns in ue5-main branch]
2023-09-14 11:39:49 -04:00
christopher waters
5dc4ce553d Removing redundant line directives from Shader Minifier.
#rb yuriy.odonnell

[CL 27872540 by christopher waters in ue5-main branch]
2023-09-14 05:20:23 -04:00
graham wihlidal
db0fea1270 Modified DirectXShaderCompiler to add a custom UserInfo container part, and check for derivative ops. The final output currently encoded the derivative ops into the ResourceDef part instead of UserInfo due to official validator issues, but I will follow up with Microsoft on a clean solution.
#rb laura.hermanns
[FYI] jason.nadro, yuriy.odonnell, rune.stubbe, brian.karis

[CL 27737340 by graham wihlidal in ue5-main branch]
2023-09-09 02:14:45 -04:00
graham wihlidal
811135f653 RDNA ISA parser improvements and fixes
[FYI] rune.stubbe, brian.karis

[CL 27697998 by graham wihlidal in ue5-main branch]
2023-09-07 20:34:04 -04:00
graham wihlidal
7f36ea203a Added two missing MIMG instructions to RDNA2_ISA.h
[CL 27697990 by graham wihlidal in ue5-main branch]
2023-09-07 20:33:59 -04:00
graham wihlidal
987e25e552 Renamed NoImplicitDerivatives -> NoDerivativeOps to also handle the explicit derivative cases, and cleaned up cflag naming
#rb rune.stubbe
[FYI] dan.elksnitis

[CL 27697881 by graham wihlidal in ue5-main branch]
2023-09-07 20:30:26 -04:00
graham wihlidal
3295fddd4f Implemented RDNA 1 & 2 ISA parser to optionally detect if a compute shader has any need for implicit derivatives (image sample, ddx, ddy, etc.). Certain shaders can be marked with CFLAG_ExplicitDerivatives, indicating the shaders prefer to handle derivatives explicitly, and this instructs the shader compiler - on supporting platforms - to parse the compiled ISA and look if any instructions requiring implicit derivatives are present. If none are found, the shader is decorated with NoImplicitDerivatives, indicating it is safe to ignore shading quads, etc..
#rb jason.nadro, luke.thatcher, dan.elksnitis
[FYI] brian.karis, rune.stubbe

[CL 27697142 by graham wihlidal in ue5-main branch]
2023-09-07 20:10:55 -04:00
louisphilippe seguin
a11435338f Fix usage of const TCHAR* in the condition of checkf or ensureMsgf and test the strings for nullptr instead
#rb trivial
#rnx

[CL 27427505 by louisphilippe seguin in ue5-main branch]
2023-08-28 17:50:38 -04:00
jeannoe morissette
02a417081c ShaderConductor: Change the default for bDisableScalarBlockLayout to TRUE so that it matches the previous behavior where it was never truly enabled. Attempt to fix build where shader alignment issues started poping up.
#rb none
#rnx

[CL 27402811 by jeannoe morissette in ue5-main branch]
2023-08-26 13:12:30 -04:00
jeannoe morissette
235b6cbfd3 VulkanRHI: Actually enable scalar_block_layout in raytracing and sm6 shaders to fix multiple alignment issues. Rename ReflectionSlot param to align with ShaderCommon code.
#rb carl.lloyd
#jira UE-171798
#rnx

[CL 27399055 by jeannoe morissette in ue5-main branch]
2023-08-25 22:19:15 -04:00
carl lloyd
34d53bc691 Added validation to ensure UAV/SRV types and UB sizes match the ones generated by the shader compiler
#rb jeannoe.morissette
#jira UE-82353

[CL 27399050 by carl lloyd in ue5-main branch]
2023-08-25 22:19:10 -04:00
jeannoe morissette
ec701f8ca4 VulkanRHI: VK RT WIP: Shader compiler changes for Vulkan RT. Add an option for extra blobs for AnyHit and Intersection shaders in HitGroup. Add option of bindless UBs for ray tracing shaders. Correctly set GRHISupportsRayTracingDispatchIndirect.
#rb Aleksander.Netzel
#jira UE-190075
#rnx

[CL 27231539 by jeannoe morissette in ue5-main branch]
2023-08-19 19:17:32 -04:00
jeannoe morissette
b070af8d82 VulkanRHI: Add quotes around the entrypoint name in the command line for cases where multiple entrypoints are possible (like RT hit group)
#rb carl.lloyd
[FYI] aleksander.netzel
#rnx

[CL 27204165 by jeannoe morissette in ue5-main branch]
2023-08-18 11:39:37 -04:00
laura hermanns
f5e5f41162 Fix removal of unused shader interpolators for DXC backends: Keep original semantic name including its index and separate it from its overridden binding slot.
#rb Dan.Elksnitis, Ben.Woodhouse
[FYI] Brian.White, Carl.Lloyd, Serge.Bernie
#jira UE-191963
#rnx

[CL 27133050 by laura hermanns in ue5-main branch]
2023-08-16 00:43:54 -04:00
dan elksnitis
9a5a1b6f74 [shaders] minor include cleanup
#rb Jason.Hoerner
#rb Laura.Hermanns

[CL 26817718 by dan elksnitis in ue5-main branch]
2023-08-03 13:40:32 -04:00
dan elksnitis
5f85b46590 [shaders] remove unnecessary passing of hlslcc flags as an option to debug info output. this was just used to pass a switch on the SCW directcompile cmdline which SCW no longer does anything with. also cleaned up some additional unnecessary code related to these flags in backends that no longer use hlslcc at all.
#rb Laura.Hermanns

[CL 26685354 by dan elksnitis in ue5-main branch]
2023-07-28 15:58:01 -04:00
charles derousiers
2365ee5308 Change D3D diagnostic buffer to output readable assert message (file/line/condition/message).
Use with check() / checkf()

#rb dan.elksnitis, christopher.waters
[FYI] yuriy.odonnell

[CL 26680005 by charles derousiers in ue5-main branch]
2023-07-28 13:55:18 -04:00
jason hoerner
7f22080814 Shader Compiler: Made FShaderCompilerDefinitions private, to reduce code publicly visible in ShaderCore.h. For now, it's just marked deprecated for 5.4, but will be hidden more generally in 5.5.
#rnx
#rb yuriy.odonnell dan.elksnitis jason.nadro

[CL 26534998 by jason hoerner in ue5-main branch]
2023-07-22 06:35:10 -04:00
tom holmes
c847704982 Mutate DDC string since platform symbol serialization data changed
Protect WriteSymbolData if the Filename is empty

#jira UE-190842
[REVIEW] [at]matt.peters
#tests cooking CitySample

[CL 26528753 by tom holmes in ue5-main branch]
2023-07-21 18:27:54 -04:00