Protect WriteSymbolData if the Filename is empty
#jira UE-190842
[REVIEW] [at]matt.peters
#tests cooking CitySample
[CL 26528753 by tom holmes in ue5-main branch]
Platforms can return a platform hash and a blob of text data to go with that hash
These are stored one / line in ShaderSymbols.info in lieu of full PDB info
Useful when full shader PDBs are impractically large, info-only is much smaller
#rb dan.elksnitis, john.huelin, serge.bernier
#tests test on cooking Lyra, various platforms, noddc and ddc
[CL 26496825 by tom holmes in ue5-main branch]
* Major define optimization involves converting map of defines to use FName keys and variant values rather than strings -- this eliminates most of the cost of string hashing, allocation, and conversion.
* Lower overhead FHashTable used instead of TMap.
* An initial map of defines can optionally be provided globally. Anything using an initial define can have its map index cached for optimized lookup when reading or writing.
Other micro-optimizations:
* Added Reserve calls for uniform buffer related maps, to eliminate map resizing / rehashing. Saved around 15% perf (after define optimizations).
* Added a map for UB lookup, instead of iterating through the linked list. Saved around 10% perf.
Non-optimization: Sort the order in which uniform buffer variable names are searched in BuildShaderFileToUniformBufferMap, to create determinism in ShaderDebug data for A/B testing (previously order was dependent on global constructor order for UB definitions, which could vary arbitrarily with unrelated changes).
#jira UE-187334
#rnx
#rb dan.elksnitis jason.nadro yuriy.odonnell
[CL 26142884 by jason hoerner in ue5-main branch]
- remove redundant options struct initialization in DumpDebugShaderData - this was resulting in losing some of the options set in the input parameter options struct
- add explicit "DumpExtendedDebugShaderData" which includes the additional debug output functionality only previously available if an IShaderFormat implementation inherited from FBaseShaderFormat
- add "AdditionalOutputs" functionality which can be used by backends to dump additional (text-based) files following the file naming rules as dictated by the options struct with less boilerplate code
- add a helper for constructing a debug filename from the debug output options struct (some backends require constructing such filenames for i.e. contents of batchfiles; the helper reduces boilerplate code in such cases)
- fix condition for old path of dumping OutputHash.txt to not redundantly dump if shader format supports independent preprocessing and preprocessed cache is enabled
#rb Massimo.Tristano
[CL 25892792 by dan elksnitis in ue5-main branch]
- add new fields to FShaderCompilerOutput which should be populated with the shader source and entry point name if modified by a backend compile process (in a format implementing the independent preprocessing API). this is intended to supercede "OptionalFinalShaderSource" for such shader formats, as well as being used as the source for preprocessed debug dumps when enabled.
- modify the PCD3D shader format to use these fields as intended (and so fix the debug output for pipelines which remove unused interpolators to match the final modified source).
#rb Laura.Hermanns
#preflight 64777f850848b7126dda194c
[CL 25706448 by dan elksnitis in ue5-main branch]
* Early out when adding uniform buffer resource entries, by checking if uniform buffer had been processed, before processing individual entries.
* Changed resource entry list from a map to an array, saving the cost of map operations adding items to it.
* Resource entries for global uniform buffers cached once at startup.
* Resource entry member names stored in an external buffer, to avoid per-entry string memory allocation overhead.
* Miscellaneous smaller optimizations -- for example, using ByHash to avoid redundant hashing and FString construction.
#jira none
#rnx
#rb jason.nadro dan.elksnitis
#preflight 646cfdbc1134ffac7034af60
[CL 25596308 by jason hoerner in ue5-main branch]
- Support structured buffers in HLSL backend of SPIRV-Cross (PR #2143).
- Integrate SPIRV-Cross fix to forward opaque types in GLSL backend (PR #2110).
- Integrate SPIRV-Cross fix to use actual field offset to validate vec4 boundary alignment (#PR 2139).
- Integrate SPIRV-Cross fix to not restrict SV_RenderTargetArrayIndex to GS and MS (#PR 2138).
- Make SV_Position decoration as implicitly invariant optional (Used to address Z-fighting issues in Vulkan and Metal; Drastically impacts vertex shader performance on certain platforms).
- Don't force DXC for all shaders when r.Substrate is enabled.
#jira UE-174555
#rb Sebastien.Hillaire, Carl.Lloyd, Dan.Elksnitis
#fyi Brian.White, Jason.Nadro, JeaNoe.Morissette, Yuriy.Odonnell, Guillaume.Abadie
#preflight 6453d3df0e33f2d51d2b032e
[CL 25353473 by Laura Hermanns in ue5-main branch]
- add new IShaderFormat API for separate preprocessing and compilation; backends can implement one or the other depending on the return value of SupportsIndependentPreprocessing
- add support for executing preprocessing in the cook process prior to job submission and constructing job input hashes based on preprocessed source (and a subset of the environment used as compile inputs). controlled by a cvar for now and disabled by default
- add a BaseShaderFormat class in ShaderCompilerCommon which implements common behaviour for output of debug data - note this function is only called for formats which support independent preprocessing, so is expected to be used only by formats which have been converted to use this API
- add new cvars for output of some additional shader debug data - 1. a txt file containing the input hash a.k.a. job cache key 2. a text file containing all diagnostic messages (errors and warnings) for the job
- minor change to how input hashes are constructed for pipeline jobs - sum hashes as 256-bit ints instead of adding to a buffer and re-hashing. faster and simpler, and also more collision resistant (sum of two well distributed hashes equally well distributed)
#rb Jason.Nadro
#rb Yuriy.ODonnell
#preflight 64512c88c86798f650b953d3
[CL 25317218 by dan elksnitis in ue5-main branch]
- add a debug info flags enum/field to indicate which shader debug outputs are desired
- change direct compile txt file mechanism to use this (more options will be added in the near future)
- rename the field on the debug info dump options which indicates that the direct compile file should be skipped to bSourceOnly (when addtional outputs are added these should be skipped as well)
#preflight 6447f6c4b208f61af876e215
#rb Jason.Nadro
#rb Massimo.Tristano
[CL 25183084 by dan elksnitis in ue5-main branch]
