Commit Graph

166 Commits

Author SHA1 Message Date
dan elksnitis
cc7c2c54f4 [shaders] shader format preprocessing cleanup & refactoring
- move uniform buffer cleanup and dead stripping into ShaderPreprocessor module's PreprocessShader function
- add "required symbols" to compiler input struct to specify additional symbols to keep during minification aside from those specified by the entrypoint; modify API such that both an entry point string and additional symbols can be specified (to avoid each backend needing to manually parse the compound RT entry point string)
- make use of ModifyShaderCompilerInput in all backends to set additional defines and required symbols on input struct up front; only use the AdditionalDefines map in cases where it's actually necessary
- remove the various per-platform defines for enabling minifier, no longer required now that this has been rolled out for all backends
- fix SCW directcompile mode; this had rotted due to pieces of the FShaderCompilerEnvironment having been added that weren't explicitly serialized to either cmdline or in the shader source. this now serializes as a base64 string written inside the USF containing all portions of the environment required for compilation (using the same serialization function as is used to write/read the SCW input file)
- use a debug flag for indicating we're in "direct compile" mode and should load the debug USF off disk, rather than the poorly named "bSkipPreprocessedCache" (this name is both inaccurate and also confusing due to the addition of the preprocessed job cache)
- modify platform "force wave32" mechanism to use a pragma directive to set a compiler define, instead of doing string replacement in the preprocessed source
- add a view version of the RT entrypoint parsing to use in preprocessing, note that other paths still need to construct fstrings due to further manipulation so keeping the FString path around too
- clean up backends manually checking the "directcompile" cmdline arg

#rb christopher.waters, Yuriy.ODonnell
#rb Chris.Waters
#rb Laura.Hermanns

[CL 30023082 by dan elksnitis in ue5-main branch]
2023-11-30 15:56:34 -05:00
daniele vettorel
e39770acde Support exporting shader symbols info for PC D3D_SM6 shaders.
The PDB name is either retrieved from the PDB blob part of DXC's output, or is generated using the shader hash that DXC reports even when no PDB is requested. We don't export PDBs because it would result in duplicate data as we use embedded PDBs.

#rb christopher.waters, Laura.Hermanns

[CL 29723169 by daniele vettorel in ue5-main branch]
2023-11-14 15:18:29 -05:00
laura hermanns
d6acdc4eab Make DXC compile arguments consistent between Vulkan backend and CompileDXC.bat script.
#jira UE-198594
#rb Dan.Elksnitis
#rnx

[CL 29569710 by laura hermanns in ue5-main branch]
2023-11-08 14:23:36 -05:00
christopher waters
6f3f63b186 Bindless Conversion Changes
- Removing the need for COMPILER_DEFINE on GetResourceFromHeap and GetSamplerFromHeap
- FShaderParameterParser constructor now requires a FPlatformConfiguration object that each shader format can configure.
- FPlatformConfiguration::GenerateBindlessAccess allows each platform to generate the code for accessing a bindless resource or sampler.
- Added EShaderParameterParserConfigurationFlags to specify support for stable constant buffers and bindless.
- Moved EBindlessParameterMode into EShaderParameterParserConfigurationFlags::BindlessUsesArrays
- While moving things around, I was able to convert some TCHAR* uses to FStringView

#jira UE-166341
#rb mihnea.balta

[CL 29533376 by christopher waters in ue5-main branch]
2023-11-07 16:26:20 -05:00
serge bernier
9ce7402b1f Support INVARIANT on platform using DXC-SPIRV
-Add a new define to wrap the output semantic that we want to be post declare as precise.
-Modify DXC-SIRV to add the NoContraction decorator on precise output semantics.
-Modify spirv_glsl to post declare as precise output semantics
  -Example: out precise float4 gl_positon; will add precise gl_position; post declaration.

[REVIEW] [at]laura.hermanns

[CL 29338529 by serge bernier in ue5-main branch]
2023-11-01 19:14:21 -04:00
laura hermanns
1daf299a10 Patch order of entry point parameters in cross-compiled HLSL of D3D11 backend using HlslParser.
#jira UE-186066
#rb Dan.Elksnitis
#rnx

[CL 29257708 by laura hermanns in ue5-main branch]
2023-10-30 19:37:05 -04:00
jason hoerner
0e18ea7e81 Shader Preprocessor: Roll TEXT macro substitution into the C preprocessor step, via custom macro callback feature, making it effectively free. Saves 3% overall on preprocess, but also represents a simplification by reducing the number of unique preprocess passes.
#jira UE-198496
#rnx
#rb dan.elksnitis yuriy.odonnell jason.nadro

[CL 29100104 by jason hoerner in ue5-main branch]
2023-10-25 16:01:11 -04:00
christopher waters
ab21e17d39 Bindless Resources
- Splitting "Bindless Resource" shader parameter types into "Bindless SRV" and "Bindless UAV" to make sure reflection/binding validation is correct.

#jira UE-166341
#rb jeannoe.morissette

[CL 28995862 by christopher waters in ue5-main branch]
2023-10-20 23:12:44 -04:00
laura hermanns
0320f2f528 D3D11 cross-compilation fixes:
- Patch HLSL to support SV_DepthGreaterEqual and SV_DepthLessEqual.
- Maintain order of shader input/output variables in cross-compiled HLSL.

#rb Jason.Nadro
#jira UE-186066, UE-192125
#rnx

[CL 28633421 by laura hermanns in ue5-main branch]
2023-10-10 15:30:19 -04:00
christopher waters
b9351620ad HlslParser
- Adding ability to 'forward declare' symbols per platform.
- Removes hardcoded symbols in HlslParser.cpp
- Converting a few array/string input types to views.

#rb dan.elksnitis

[CL 28494936 by christopher waters in ue5-main branch]
2023-10-05 08:26:39 -04:00
dan elksnitis
4515e3b5f4 [shaders] add debug dump option to output "detailed" shader source, including the output of the preprocessing step unmodified, the stripped version used for hashing, and whatever the final source passed to the compiler is (can differ from the stripped version if further modifications are applied as part of the compile step). this is only done if the preprocessed job cache is enabled; otherwise (and if this cvar is disabled) the single output will either be the compiler-step-modified source if it exists, or the output of the preprocessing step.
#rb Jason.Hoerner

[CL 28274493 by dan elksnitis in ue5-main branch]
2023-09-27 11:02:18 -04:00
christopher waters
bb0a274b8c Reworking UniformBuffer code generation to not use an intermediate struct
- When generating Uniform Buffer HLSL code, a 'static const struct {} Name' was created which has been causing compiler issues off and on for years.
- When RemoveUniformBuffersFromSource removed said struct, it parsed the struct's structure to figure out how to translate UniformBuffer.Member into UniformBuffer_Member in code that accessed Uniform Buffers.
- Now, instead of a struct, a "UniformBuffer <Name>" declaration is created and filled with mappings from "UniformBuffer.Member" into the correct code.
- This allows ConstantBuffer objects to be used differently from cbuffer where the former generates "UniformBuffer.Member" and the latter generates "UniformBuffer_Member"
- This also allows resources to always be translated to "UniformBuffer_Member"
- Shader define 'PLATFORM_SUPPORTS_UB_STRUCT' is now 'PLATFORM_SUPPORTS_CONSTANTBUFFER_OBJECT' and will be set on shader formats that support ConstantBuffer<> objects.
- Removed lots of unused code from BindlessResources.ush

#jira UE-195103
#rb dan.elksnitis, jeannoe.morissette

[CL 28058373 by christopher waters in ue5-main branch]
2023-09-20 17:33:36 -04:00
laura hermanns
f9b3974ab9 Platform-specific DXC improvements:
- Avoid explicit FMA instructions in platform-specific GLSL to let backend-compiler decide when to use them.
- Fix code refelction when using SPIR-V directly with backend-compiler.
- Define preset of SPIR-V optimization passes to reduce nested expressions "preset(relax-nested-expr)".

#rb Serge.Bernier, Brian.White
[FYI] Graeme.Thornton, Jason.Nadro, Carl.Lloyd, Florin.Pascu, Michael.WAnderson
#jira UE-195236, UE-194869

[CL 27994871 by laura hermanns in ue5-main branch]
2023-09-19 10:25:17 -04:00
laura hermanns
d7fff598d5 Allow passing custom SPIV-V optimization passes to ShaderConductor. This allows us to control more fine-grained what optimization passes work best for certain render passes on different platforms.
#rb Serge.Bernier
[FYI] Carl.Lloyd, JeanNoe.Morissette
#rnx

[CL 27880727 by laura hermanns in ue5-main branch]
2023-09-14 11:39:49 -04:00
graham wihlidal
db0fea1270 Modified DirectXShaderCompiler to add a custom UserInfo container part, and check for derivative ops. The final output currently encoded the derivative ops into the ResourceDef part instead of UserInfo due to official validator issues, but I will follow up with Microsoft on a clean solution.
#rb laura.hermanns
[FYI] jason.nadro, yuriy.odonnell, rune.stubbe, brian.karis

[CL 27737340 by graham wihlidal in ue5-main branch]
2023-09-09 02:14:45 -04:00
graham wihlidal
987e25e552 Renamed NoImplicitDerivatives -> NoDerivativeOps to also handle the explicit derivative cases, and cleaned up cflag naming
#rb rune.stubbe
[FYI] dan.elksnitis

[CL 27697881 by graham wihlidal in ue5-main branch]
2023-09-07 20:30:26 -04:00
graham wihlidal
3295fddd4f Implemented RDNA 1 & 2 ISA parser to optionally detect if a compute shader has any need for implicit derivatives (image sample, ddx, ddy, etc.). Certain shaders can be marked with CFLAG_ExplicitDerivatives, indicating the shaders prefer to handle derivatives explicitly, and this instructs the shader compiler - on supporting platforms - to parse the compiled ISA and look if any instructions requiring implicit derivatives are present. If none are found, the shader is decorated with NoImplicitDerivatives, indicating it is safe to ignore shading quads, etc..
#rb jason.nadro, luke.thatcher, dan.elksnitis
[FYI] brian.karis, rune.stubbe

[CL 27697142 by graham wihlidal in ue5-main branch]
2023-09-07 20:10:55 -04:00
jeannoe morissette
02a417081c ShaderConductor: Change the default for bDisableScalarBlockLayout to TRUE so that it matches the previous behavior where it was never truly enabled. Attempt to fix build where shader alignment issues started poping up.
#rb none
#rnx

[CL 27402811 by jeannoe morissette in ue5-main branch]
2023-08-26 13:12:30 -04:00
carl lloyd
34d53bc691 Added validation to ensure UAV/SRV types and UB sizes match the ones generated by the shader compiler
#rb jeannoe.morissette
#jira UE-82353

[CL 27399050 by carl lloyd in ue5-main branch]
2023-08-25 22:19:10 -04:00
jeannoe morissette
ec701f8ca4 VulkanRHI: VK RT WIP: Shader compiler changes for Vulkan RT. Add an option for extra blobs for AnyHit and Intersection shaders in HitGroup. Add option of bindless UBs for ray tracing shaders. Correctly set GRHISupportsRayTracingDispatchIndirect.
#rb Aleksander.Netzel
#jira UE-190075
#rnx

[CL 27231539 by jeannoe morissette in ue5-main branch]
2023-08-19 19:17:32 -04:00
dan elksnitis
5f85b46590 [shaders] remove unnecessary passing of hlslcc flags as an option to debug info output. this was just used to pass a switch on the SCW directcompile cmdline which SCW no longer does anything with. also cleaned up some additional unnecessary code related to these flags in backends that no longer use hlslcc at all.
#rb Laura.Hermanns

[CL 26685354 by dan elksnitis in ue5-main branch]
2023-07-28 15:58:01 -04:00
charles derousiers
2365ee5308 Change D3D diagnostic buffer to output readable assert message (file/line/condition/message).
Use with check() / checkf()

#rb dan.elksnitis, christopher.waters
[FYI] yuriy.odonnell

[CL 26680005 by charles derousiers in ue5-main branch]
2023-07-28 13:55:18 -04:00
jason hoerner
7f22080814 Shader Compiler: Made FShaderCompilerDefinitions private, to reduce code publicly visible in ShaderCore.h. For now, it's just marked deprecated for 5.4, but will be hidden more generally in 5.5.
#rnx
#rb yuriy.odonnell dan.elksnitis jason.nadro

[CL 26534998 by jason hoerner in ue5-main branch]
2023-07-22 06:35:10 -04:00
tom holmes
ece26e8d8b Support for slim ShaderSymbols.info / r.Shaders.SymbolsInfo=1
Platforms can return a platform hash and a blob of text data to go with that hash
These are stored one / line in ShaderSymbols.info in lieu of full PDB info
Useful when full shader PDBs are impractically large, info-only is much smaller

#rb dan.elksnitis, john.huelin, serge.bernier
#tests test on cooking Lyra, various platforms, noddc and ddc

[CL 26496825 by tom holmes in ue5-main branch]
2023-07-20 16:51:51 -04:00
christopher waters
4e96f4193d Removing UE_ENABLE_INCLUDE_ORDER_DEPRECATED_IN_5_0 and UE_ENABLE_INCLUDE_ORDER_DEPRECATED_IN_5_1 including all uses in the engine.
[CL 26259545 by christopher waters in ue5-main branch]
2023-06-27 11:54:23 -04:00