Commit Graph

334 Commits

Author SHA1 Message Date
serge bernier
9ce7402b1f Support INVARIANT on platform using DXC-SPIRV
-Add a new define to wrap the output semantic that we want to be post declare as precise.
-Modify DXC-SIRV to add the NoContraction decorator on precise output semantics.
-Modify spirv_glsl to post declare as precise output semantics
  -Example: out precise float4 gl_positon; will add precise gl_position; post declaration.

[REVIEW] [at]laura.hermanns

[CL 29338529 by serge bernier in ue5-main branch]
2023-11-01 19:14:21 -04:00
laura hermanns
1daf299a10 Patch order of entry point parameters in cross-compiled HLSL of D3D11 backend using HlslParser.
#jira UE-186066
#rb Dan.Elksnitis
#rnx

[CL 29257708 by laura hermanns in ue5-main branch]
2023-10-30 19:37:05 -04:00
jason hoerner
0e18ea7e81 Shader Preprocessor: Roll TEXT macro substitution into the C preprocessor step, via custom macro callback feature, making it effectively free. Saves 3% overall on preprocess, but also represents a simplification by reducing the number of unique preprocess passes.
#jira UE-198496
#rnx
#rb dan.elksnitis yuriy.odonnell jason.nadro

[CL 29100104 by jason hoerner in ue5-main branch]
2023-10-25 16:01:11 -04:00
jason hoerner
4bb44a53c7 Shader Preprocessor: Compact commented out uniform buffer declarations out of the source code in CleanupUniformBufferCode. Saves 9% of overall preprocess time by avoiding comment scanning in minifier parse step. Also took this opportunity to remove the unused original CleanupUniformBufferCode implementation -- the new version has been stable, so we don't need a CVar toggle any more.
#rnx
#rb yuriy.odonnell jason.nadro

[CL 29095286 by jason hoerner in ue5-main branch]
2023-10-25 13:58:44 -04:00
christopher waters
ab21e17d39 Bindless Resources
- Splitting "Bindless Resource" shader parameter types into "Bindless SRV" and "Bindless UAV" to make sure reflection/binding validation is correct.

#jira UE-166341
#rb jeannoe.morissette

[CL 28995862 by christopher waters in ue5-main branch]
2023-10-20 23:12:44 -04:00
laura hermanns
1f90f8d0a2 Extend HlslParser to support Substrate:
- Support struct forward declarations.
- Support static member functions.
- Support scoped function definitions.

#jira UE-186066
#rb Jason.Nadro
[FYI] Sebastien.Hillaire, Christopher.Waters
#rnx

[CL 28959874 by laura hermanns in ue5-main branch]
2023-10-20 06:51:39 -04:00
matt peters
e53bae0868 MPCook: Add global function UE::GetMultiprocessId to handle the commandline parsing of MultiprocessId for CookWorkers.
#rnx
#rb Zousar.Shaker

[CL 28938402 by matt peters in ue5-main branch]
2023-10-19 18:18:01 -04:00
yuriy odonnell
57397e6c36 Add CPU profiler scope to UE::ShaderCompilerCommon::RemoveDeadCode
#rb none

[CL 28744091 by yuriy odonnell in ue5-main branch]
2023-10-13 04:02:29 -04:00
yuriy odonnell
355a38075e Fix compile error on linux/mac due to indexing arrays with char type
#rb none

[CL 28699927 by yuriy odonnell in ue5-main branch]
2023-10-11 23:46:03 -04:00
yuriy odonnell
dd72ffe26c ShaderMinifier - Optimize BuildLineBreakMap, SkipUntilNonIdentifierCharacter, ExtractIdentifiers and few other bits
* Generate line directive map during parsing
* Use xxhash instead of cityhash
* Avoid FStringView operator [] in ExtractIdentifiers
* Use TArray Reset instead of Empty where appropriate
* Vectorize SkipUntilNonIdentifierCharacter
* Vectorize ExtractBlock
* Vectorize line ending search
* Use FString directly instead of FStringBuilderBase for the final output to avoid a copy

#rb christopher.waters
#rb jason.hoerner

[CL 28699609 by yuriy odonnell in ue5-main branch]
2023-10-11 23:32:28 -04:00
christopher waters
0403f2caa0 Always set the HLSL version and don't assume the default is 2018.
#rb David.Harvey, Yuriy.Odonnell

[CL 28692390 by christopher waters in ue5-main branch]
2023-10-11 19:52:44 -04:00
laura hermanns
0320f2f528 D3D11 cross-compilation fixes:
- Patch HLSL to support SV_DepthGreaterEqual and SV_DepthLessEqual.
- Maintain order of shader input/output variables in cross-compiled HLSL.

#rb Jason.Nadro
#jira UE-186066, UE-192125
#rnx

[CL 28633421 by laura hermanns in ue5-main branch]
2023-10-10 15:30:19 -04:00
christopher waters
0121d81804 Fixing crash that was hidden by hardcoded namespaces.
[CL 28495033 by christopher waters in ue5-main branch]
2023-10-05 08:27:30 -04:00
christopher waters
b9351620ad HlslParser
- Adding ability to 'forward declare' symbols per platform.
- Removes hardcoded symbols in HlslParser.cpp
- Converting a few array/string input types to views.

#rb dan.elksnitis

[CL 28494936 by christopher waters in ue5-main branch]
2023-10-05 08:26:39 -04:00
yuriy odonnell
eb793d3fb5 ShaderMinifier - Use inline implementations of string view Equals and StartsWith
* Also use TEXTVIEW instead of TEXT where possible
* ~1.5x speedup

#rb dan.elksnitis

[CL 28494884 by yuriy odonnell in ue5-main branch]
2023-10-05 08:26:07 -04:00
jason hoerner
bca274f611 Shader Compiler: CleanupUniformBufferCodeNew minor fix to account for escaped quotes in text. Continue string parsing if character before quote is a backslash. Note that there are no escaped quotes in any existing .usf or .ush file at the moment, so I don't think the bug should have caused any issues.
#rnx
#rb dan.elksnitis jason.nadro

[CL 28494850 by jason hoerner in ue5-main branch]
2023-10-05 08:25:49 -04:00
christopher waters
9b84b8accd Fixing enum handling in the shader minifier. Since the enum elements weren't marked as reachable identifiers, the minifier was considering the whole enum block unused.
#rb yuriy.odonnell

[CL 28438918 by christopher waters in ue5-main branch]
2023-10-03 19:32:26 -04:00
yuriy odonnell
d635588f1b ShaderMinifier - Don't skip keyword blocks when building ChunksByIdentifier (temporary workaround for an issue introduced in 28384399)
#rb christopher.waters

[CL 28417897 by yuriy odonnell in ue5-main branch]
2023-10-03 12:08:28 -04:00
yuriy odonnell
86d5bb7306 ShaderMinifier - Add basic support for global operator overloads
#rb christopher.waters

[CL 28417810 by yuriy odonnell in ue5-main branch]
2023-10-03 12:06:47 -04:00
christopher waters
c1487a0042 Adding typedef support to ShaderMinifier.
- Before this, every typedef and the code using them was being included in the final source.
- This was primarily affecting Bindless Resources where all bindless resources were included in the minified code even if they weren't used.

#rb yuriy.odonnell

[CL 28365749 by christopher waters in ue5-main branch]
2023-09-30 11:10:06 -04:00
jason hoerner
6f67025ccc Shader Compiler: Optimized version of CleanupUniformBufferCode, called during preprocessing. New version is 3.8x faster, producing a 19% overall speedup in ConditionalPreprocessShader. Processes the shader in a single pass, scanning for potential compound identifiers (symbol tokens connected by dots and optional whitespace), then checking if they match known uniform buffer members. A few optimizations improve the speed of checking if an identifier matches, including an early out based on identifier length and start character, followed by organizing uniform buffers and their members by length, requiring only a small subset of members to be tested. Parsing uniform buffer definitions was also optimized to avoid memory allocation.
Doesn't produce bit identical preprocessed output, as it compacts whitespace from compound identifiers before testing them, whereas the original implementation compacts whitespace after matching.  Compacting first makes testing more efficient and simpler, since it doesn't need to take into account whitespace.  For A/B validation, I did add temporary debug code that stored and reverted un-matched identifiers, mimicking the old behavior, and it was identical across the board.

#rnx
#rb dan.elksnitis chris.waters jason.nadro

[CL 28313553 by jason hoerner in ue5-main branch]
2023-09-28 07:08:47 -04:00
dan elksnitis
4515e3b5f4 [shaders] add debug dump option to output "detailed" shader source, including the output of the preprocessing step unmodified, the stripped version used for hashing, and whatever the final source passed to the compiler is (can differ from the stripped version if further modifications are applied as part of the compile step). this is only done if the preprocessed job cache is enabled; otherwise (and if this cvar is disabled) the single output will either be the compiler-step-modified source if it exists, or the output of the preprocessing step.
#rb Jason.Hoerner

[CL 28274493 by dan elksnitis in ue5-main branch]
2023-09-27 11:02:18 -04:00
christopher waters
7f66db7ffc Shader preprocessing optimizations:
- Removing system calls from UniformBuffer parsing and processing.
- When searching for uses of a UniformBuffer, start after the end of the UniformBuffer declaration.
- Simplifying the code around UniformBuffer reference cleanup
- Cuts time in CleanupUniformBufferCode by 50-75% based on how large the shader is.

#jira UE-195103
#rb dan.elksnitis, jason.hoerner

[CL 28172021 by christopher waters in ue5-main branch]
2023-09-23 05:43:20 -04:00
christopher waters
bb0a274b8c Reworking UniformBuffer code generation to not use an intermediate struct
- When generating Uniform Buffer HLSL code, a 'static const struct {} Name' was created which has been causing compiler issues off and on for years.
- When RemoveUniformBuffersFromSource removed said struct, it parsed the struct's structure to figure out how to translate UniformBuffer.Member into UniformBuffer_Member in code that accessed Uniform Buffers.
- Now, instead of a struct, a "UniformBuffer <Name>" declaration is created and filled with mappings from "UniformBuffer.Member" into the correct code.
- This allows ConstantBuffer objects to be used differently from cbuffer where the former generates "UniformBuffer.Member" and the latter generates "UniformBuffer_Member"
- This also allows resources to always be translated to "UniformBuffer_Member"
- Shader define 'PLATFORM_SUPPORTS_UB_STRUCT' is now 'PLATFORM_SUPPORTS_CONSTANTBUFFER_OBJECT' and will be set on shader formats that support ConstantBuffer<> objects.
- Removed lots of unused code from BindlessResources.ush

#jira UE-195103
#rb dan.elksnitis, jeannoe.morissette

[CL 28058373 by christopher waters in ue5-main branch]
2023-09-20 17:33:36 -04:00
tom holmes
0c53e701a0 Improve ShaderSymbols.info file generation in multicook scenarios
Dedupe info where multiple workers built same shader

#rb Ben.Woodhouse
#tests local runs, compare of farm built files attached in bug

[CL 27995633 by tom holmes in ue5-main branch]
2023-09-19 10:46:28 -04:00