resubmit with following fixes:
- static analysis error which caught an >=0 check on a uint64 which should have been >0
- fix for an inverted guard on multiprocess cook sending bytecode to director (was only sending code across if empty instead of non-empty)
- fix for uninitialized padding in the FShaderCodeResource::FHeader struct causing nondeterministic puts
- fix for incorrect size passed to job cache hashing on receiving buffers from DDC
#rb Devin.Doucette, Laura.Hermanns, Zousar.Shaker
#lockdown Marc.Audy
[CL 36754792 by dan elksnitis in 5.5 branch]
[FYI] dan.elksnitis
Original CL Desc
-----------------------------------------------------------------
[shaders] modify FShaderCode finalize to create a FSharedBuffer object, and modify all downstream uses of shader code to re-use this buffer (job cache, pushes to DDC, shader maps, and shader library). This reduces total amount of LLM tracked memory allocated at the end of a cold Lyra PS4 cook by about ~350MB; impact likely much larger for cooks of larger projects.
#rb Devin.Doucette, Zousar.Shaker
#lockdown Marc.Audy
resubmit with SA+MP cook fix
[CL 36747522 by dan elksnitis in 5.5 branch]
[FYI] dan.elksnitis
Original CL Desc
-----------------------------------------------------------------
[shaders] modify FShaderCode finalize to create a FSharedBuffer object, and modify all downstream uses of shader code to re-use this buffer (job cache, pushes to DDC, shader maps, and shader library). This reduces total amount of LLM tracked memory allocated at the end of a cold Lyra PS4 cook by about ~350MB; impact likely much larger for cooks of larger projects.
#rb Zousar.Shaker
#lockdown marc.audy
[CL 36440265 by dan elksnitis in 5.5 branch]
Added a comment to clarify that TNonRelocatableInlineAllocator is itself non-trivially-relocatable, not that it's for non-trivially-relocatable elements.
#rb devin.doucette
[CL 35869531 by steve robb in ue5-main branch]
#rnx
[FYI] ben.clayton
Original CL Desc
-----------------------------------------------------------------
[AutoRTFM] Migrate more critical sections to using the transactionally safe variants.
#rb neil.henning
[CL 34914957 by alex kahn in ue5-main branch]
Info is now stored in shader map in editor only data alongside platform data
Includes shader bump
#jira UE-218748
#rb dan.elksnitis
#tests local tests on Lyra cooking and DDC-Verify
[FYI] Zousar.Shaker
[CL 34906103 by tom holmes in ue5-main branch]
- add strings as an option for shader statistics, and a flags member/option to hide specific shader statistics from UI (so we can use statistics for other internal usage)
- add debug output for shader platform hashes (use to track down cases of shader job output which has identical bytecode but differing metadata), using the above to record a stat containing the shader hash computed by the compiler. currently only implemented for a subset of platforms
- fix a bug in shader diagnostic remapping for cache hits - the value stored in the cache/DDC for a job should contain non-remapped diagnostics (so cache hits can properly remap them to their version of the unstripped shader code). this also slightly improves deduplication within the job cache/DDC. fixed by storing the job output in the cache/DDC before running the job oncomplete callback (which performs the remapping)
- cleanup dxc hash retrieval code to use actual type instead of hacking around with an offset into the opaque blob
#rb Jason.Nadro
[CL 34864220 by dan elksnitis in ue5-main branch]
- Moving various UB booleans into a flags enum.
- UB booleans could not be reasonably deprecated without incurring memory overhead, so this will break custom code that uses them.
- Adding UB flag to force the shader compilers to generate reflection for the UB members which are normally excluded from reflection.
- Adding UB flag that tells MeshCommands that a UB will be bound during pass drawing and that it doesn't need to be set via MDCs.
- New flags are not used in this CL, they are prerequisites for subsequent, larger changes.
#rb jeannoe.morissette
[CL 34356503 by christopher waters in ue5-main branch]
Interface variable scalar replacement pass rewrite (added as a new pass into spirv-opt, will replace existing one in future).
Things done so far:
- Rewritten to properly handle non-aggregate types and other tricky cases (both for Patch decorated variables and not)
- Matrix scalarization is now optional.
- More test cases added
- Path is now exposed as a command-line flag
- Fixed https://github.com/KhronosGroup/SPIRV-Tools/issues/4795
Few shaders contained loops, which were not unrolled, leading to the scalarization pass to fail. Loops were not unrolled due to "complex" condition expression. Those are changed to be accepted by the loop unroller (induction variable is the only lhs in the condition expression now).
This new pass is now always enabled for both Vulkan/GLES shaders for Android.
Multiple location remapping passes were removed: one was overriding another one results + not needed now due to arrays being removed by the scalarization pass.
#rb carl.lloyd, denys.mentiei
[FYI] Dmitriy.Dyomin, Allan.Bentham
[CL 34122278 by laura hermanns in ue5-main branch]
[FYI] denys.mentiei
Original CL Desc
-----------------------------------------------------------------
[Shaders] SPIRV-Tools pass rewrite
Interface variable scalar replacement pass rewrite (added as a new pass into spirv-opt, will replace existing one in future).
Things done so far:
- Rewritten to properly handle non-aggregate types and other tricky cases (both for Patch decorated variables and not)
- Matrix scalarization is now optional.
- More test cases added
- Path is now exposed as a command-line flag
- Fixed https://github.com/KhronosGroup/SPIRV-Tools/issues/4795
Few shaders contained loops, which were not unrolled, leading to the scalarization pass to fail. Loops were not unrolled due to "complex" condition expression. Those are changed to be accepted by the loop unroller (induction variable is the only lhs in the condition expression now).
This new pass is now always enabled for both Vulkan/GLES shaders for Android.
Multiple location remapping passes were removed: one was overriding another one results + not needed now due to arrays being removed by the scalarization pass.
#rb carl.lloyd, Laura.Hermanns
[FYI] Dmitriy.Dyomin, Allan.Bentham
[CL 34107760 by laura hermanns in ue5-main branch]
Interface variable scalar replacement pass rewrite (added as a new pass into spirv-opt, will replace existing one in future).
Things done so far:
- Rewritten to properly handle non-aggregate types and other tricky cases (both for Patch decorated variables and not)
- Matrix scalarization is now optional.
- More test cases added
- Path is now exposed as a command-line flag
- Fixed https://github.com/KhronosGroup/SPIRV-Tools/issues/4795
Few shaders contained loops, which were not unrolled, leading to the scalarization pass to fail. Loops were not unrolled due to "complex" condition expression. Those are changed to be accepted by the loop unroller (induction variable is the only lhs in the condition expression now).
This new pass is now always enabled for both Vulkan/GLES shaders for Android.
Multiple location remapping passes were removed: one was overriding another one results + not needed now due to arrays being removed by the scalarization pass.
#rb carl.lloyd, Laura.Hermanns
[FYI] Dmitriy.Dyomin, Allan.Bentham
[CL 34100366 by denys mentiei in ue5-main branch]
- TCHAR* arguments were being used to find elements in a TMap<FString,..> which creates temporary memory allocations.
- We can use FStringView with FindByHash/RemoveByHash to prevent temporary allocations.
- Adding FShaderParameterMap::FindAndRemoveParameterAllocation as a shortcut for Find+Remove (which Uniform Buffer member handling does).
- Fixing a few locations that were going FString->TCHAR*->FString.
#rb Laura.Hermanns
[CL 34065428 by christopher waters in ue5-main branch]
[FYI] bob.tellez
Original CL Desc
-----------------------------------------------------------------
[Backout] - CL33671805
[FYI] stu.mckenna
Original CL Desc
-----------------------------------------------------------------
- Fix for AddUsedInputMembers having inverted source / destination for arrays
#rb Laura.Hermanns
[CL 33726075 by stu mckenna in ue5-main branch]
[FYI] stu.mckenna
Original CL Desc
-----------------------------------------------------------------
- Fix for AddUsedInputMembers having inverted source / destination for arrays
#rb Laura.Hermanns
[CL 33681068 by bob tellez in ue5-main branch]
This adds "-Wno-unknown-attributes" to DXC backends and platform specific compiler argument to their respective backend.
#rnx
#rb Yuriy.ODonnell
[CL 33437776 by laura hermanns in ue5-main branch]
