resubmit with following fixes:
- static analysis error which caught an >=0 check on a uint64 which should have been >0
- fix for an inverted guard on multiprocess cook sending bytecode to director (was only sending code across if empty instead of non-empty)
- fix for uninitialized padding in the FShaderCodeResource::FHeader struct causing nondeterministic puts
- fix for incorrect size passed to job cache hashing on receiving buffers from DDC
#rb Devin.Doucette, Laura.Hermanns, Zousar.Shaker
#lockdown Marc.Audy
[CL 36754792 by dan elksnitis in 5.5 branch]
Save up to 16MB on some platforms.
#rb daniele.vettorel, danny.couture, Kenzo.Terelst
#tests 66d8b4154a47dde1dd226334
#ushell-cherrypick of 36036431 by serge.bernier (first of 2 submits)
[CL 36747837 by serge bernier in 5.5 branch]
[FYI] dan.elksnitis
Original CL Desc
-----------------------------------------------------------------
[shaders] modify FShaderCode finalize to create a FSharedBuffer object, and modify all downstream uses of shader code to re-use this buffer (job cache, pushes to DDC, shader maps, and shader library). This reduces total amount of LLM tracked memory allocated at the end of a cold Lyra PS4 cook by about ~350MB; impact likely much larger for cooks of larger projects.
#rb Devin.Doucette, Zousar.Shaker
#lockdown Marc.Audy
resubmit with SA+MP cook fix
[CL 36747522 by dan elksnitis in 5.5 branch]
[FYI] dan.elksnitis
Original CL Desc
-----------------------------------------------------------------
[shaders] modify FShaderCode finalize to create a FSharedBuffer object, and modify all downstream uses of shader code to re-use this buffer (job cache, pushes to DDC, shader maps, and shader library). This reduces total amount of LLM tracked memory allocated at the end of a cold Lyra PS4 cook by about ~350MB; impact likely much larger for cooks of larger projects.
#rb Zousar.Shaker
#lockdown marc.audy
[CL 36440265 by dan elksnitis in 5.5 branch]
Previously, it was looking in the base directory only.
However, when shader library is split between different install bundles, chunks might be scattered over multiple directories.
It actually happens with .metallibs, which are located in different UFS folders now (which is fine).
Refer to CL 35423329 to see why.
The fix is to look through the mounted chunk folder, so shader library picks that.
#rb anderson.ramos, Arciel.Rekman
#rnx
[CL 36227046 by denys mentiei in 5.5 branch]
- create utility classes (serialization context objects) which allow serialization of shader code blobs independently of "shader object" (i.e. whatever type of thing we're pushing to the cache, shader job or shader map) structure
- commonize bytecode format in cache entries for shadermap and shader DDC (FShaderCodeResource struct)
- rework per-shader caching to use the serialization helpers to separate serialization of job output struct data and bytecode
- rework material and global shadermap DDC serialization to use the serialization helpers to separate serialization of shadermap structural data and bytecode
#rb Laura.Hermanns, Zousar.Shaker
#jira UE-196556
(resubmit with monolithic build fix)
[CL 36153164 by dan elksnitis in 5.5 branch]
- We blindly iterate _all_ content plugins and try to load a shader library for them. Not all plugins will have shaders/materials so this results in too much log spam.
- This logging has caused confusion frequently so downgrading it will help.
[CL 34871521 by jason nadro in ue5-main branch]
- Credits to Rex Hill for working out the mechanics of the leak.
#rb Rex.Hill
[REVIEW] [at]Rex.Hill, [at]Daniele.Vetorel
[CL 32331074 by arciel rekman in ue5-main branch]
if (!IsRunningWithPakFile() && ShouldLookForLooseCookedChunks())
to not affect performance in Test and Shipping builds that are using pak files.
#rb Yuriy.ODonnell
[FYI] Arciel.Rekman
#jira UE-209658
[CL 32254188 by daniele pieroni in ue5-main branch]
[FYI] dan.elksnitis
Original CL Desc
-----------------------------------------------------------------
[shaders]
- store shadertype associations for each shader in a shadermap in the editor only data
- output a debug artifact containing per-shadertype permutation/memory stats; note that this is not directly representative of final shader memory usage since it doesn't account for shader library deduplication or shader library chunk re-duplication; it is only intended to be used as a tool for tracking/identifying shader growth due to added/modified shadertypes
- remove the old "dumpshadercodestats" path from the shader library, as well as the stats tracking "unique" shaders and shader memory; the former is no longer used and the latter does not correctly account for library chunking
- bump shader version due to shadermap editoronly data change
#rb Arciel.Rekman
[CL 31780302 by bob tellez in ue5-main branch]
We expose the previously private, and unused HasScriptObjectsChunk method to the IODispatcher, as multiple codepaths are relying on this path to determine if the IoDispatcher is being used at runtime to load from chunks.
#jira UE-206417
#rb Per.Larsson
[FYI] Francis.Hurteau
[CL 31615213 by kevin macaulayvacher in ue5-main branch]
- store shadertype associations for each shader in a shadermap in the editor only data
- output a debug artifact containing per-shadertype permutation/memory stats; note that this is not directly representative of final shader memory usage since it doesn't account for shader library deduplication or shader library chunk re-duplication; it is only intended to be used as a tool for tracking/identifying shader growth due to added/modified shadertypes
- remove the old "dumpshadercodestats" path from the shader library, as well as the stats tracking "unique" shaders and shader memory; the former is no longer used and the latter does not correctly account for library chunking
- bump shader version due to shadermap editoronly data change
#rb Arciel.Rekman
[CL 31423534 by dan elksnitis in ue5-main branch]
