- Remove the condition causing SCW to automatically exit after a single job if HLSLcc was used. This doesn't appear to be beneficial; HLSLcc uses a lot of memory but does not appear to leak it, so we're wasting time spinning up new processes unnecessarily here. This allows us to deprecated the associated field in the compiler output and function in IShaderFormat.
- Strip out compiler invocations stats; this was not accurate in some cases and it was decided after discussion that at this point the effort to fix the problems was not worth the utility we get from it.
#rb Jason.Nadro
#rb Laura.Hermanns
[CL 26256807 by dan elksnitis in 5.3 branch]
* Early out when adding uniform buffer resource entries, by checking if uniform buffer had been processed, before processing individual entries.
* Changed resource entry list from a map to an array, saving the cost of map operations adding items to it.
* Resource entries for global uniform buffers cached once at startup.
* Resource entry member names stored in an external buffer, to avoid per-entry string memory allocation overhead.
* Miscellaneous smaller optimizations -- for example, using ByHash to avoid redundant hashing and FString construction.
#jira none
#rnx
#rb jason.nadro dan.elksnitis
#preflight 646cfdbc1134ffac7034af60
[CL 25596308 by jason hoerner in ue5-main branch]
- add new IShaderFormat API for separate preprocessing and compilation; backends can implement one or the other depending on the return value of SupportsIndependentPreprocessing
- add support for executing preprocessing in the cook process prior to job submission and constructing job input hashes based on preprocessed source (and a subset of the environment used as compile inputs). controlled by a cvar for now and disabled by default
- add a BaseShaderFormat class in ShaderCompilerCommon which implements common behaviour for output of debug data - note this function is only called for formats which support independent preprocessing, so is expected to be used only by formats which have been converted to use this API
- add new cvars for output of some additional shader debug data - 1. a txt file containing the input hash a.k.a. job cache key 2. a text file containing all diagnostic messages (errors and warnings) for the job
- minor change to how input hashes are constructed for pipeline jobs - sum hashes as 256-bit ints instead of adding to a buffer and re-hashing. faster and simpler, and also more collision resistant (sum of two well distributed hashes equally well distributed)
#rb Jason.Nadro
#rb Yuriy.ODonnell
#preflight 64512c88c86798f650b953d3
[CL 25317218 by dan elksnitis in ue5-main branch]
#fyi dan.elksnitis
Original CL Desc
-----------------------------------------------------------------
[shaders] unify handling of single job bSucceeded flag; wasn't getting set properly when not using workers
#rb Yuriy.ODonnell
#rb Laura.Hermanns
#rb Jason.Nadro
#preflight 63d2e011d21dbe1d29be43c1
[CL 23889694 by dan elksnitis in ue5-main branch]
- move compile job types into RenderCore and use them in SCW
- using the above factor out a helper function for pipeline job compilation to further reduce code duplication
- make flags indicating job success/failure conditions part of the SCW output data rather than evaluating them when reading the output (so they can be evaluated once in the helper function mentioned above)
- modify debug worker input dumps to write a single file for combined-pipeline compilation rather than one file per stage
#rb Yuriy.ODonnell
#preflight 63d1725d574ab9cae4c93d76
[CL 23868225 by dan elksnitis in ue5-main branch]
this has much potential for misuse, and is generally incompatible with the goal of in-process multithreaded compilation
#rb Jason.Nadro
#rb Yuriy.ODonnell
#preflight 63c94ba902024f93d83dfbbc
[CL 23772691 by dan elksnitis in ue5-main branch]
[FYI] Laura.Hermanns
Original CL Desc
-----------------------------------------------------------------
Add host machine name to OOM report in DXC backend and refactor global SCW error-code handling.
#rb Jason.Nadro, Dan.Elksnitis, Yuriy.Odonnell, Arciel.Rekman
#preflight 638e610c255f07df8ea212f4
#rnx
[CL 23421737 by kate ellis in ue5-main branch]
- allow 0 TimeToLive to be specified explicitly on commandline (useful when debugging single-job debug worker input files, without this it keeps running the same job repeatedly) and modify single job worker input commandline txt file to use this instead of 0.5f
- change config-conditional behaviour serializing to memory instead of file on disk to be conditional on an optional cmdline argument instead, and move file copy into the block that does the file writing (otherwise it repeatedly tries and fails to copy the nonexistent file)
#preflight 6377fed033774509008c42c6
#rb Jason.Nadro
[CL 23234934 by dan elksnitis in ue5-main branch]
- fix collisions in debug dumps for material shaders - name the folder based on the material instance name (if building a material instance) instead of just the base material name, also hash the asset path and static parameters to differentiate materials with the same name and different path and different static permutations
- add a debug dump option for writing single-job ShaderCompileWorker input files to allow debugging SCW in normal operation mode (including a separate txt file containing commandline args for SCW to do so)
- use path concatenation operator instead of + in shadercompileworker when constructing input/output file paths
#preflight 6362a45d6000216d2e15cd5b
#rb Christopher.Waters
#rb Rolando.Caloca
#rb Yuriy.ODonnell
[CL 22920594 by dan elksnitis in ue5-main branch]
- Focused around moving GlobalBeginCompileShader and friends.
- ModifyCompilationEnvironment and ValidateCompiledResult now only compiled in Editor builds.
- Measured 0.5MB to 1.0MB elf size reduction depending on platform.
#jira none
#rb jason.nadro, arciel.rekman, florin.pascu
#preflight 63613f992b5338aceb442902
[CL 22890964 by christopher waters in ue5-main branch]
- each platform compilation request is now wrapped in __try/__except rather than a single exception handling block at the top level of the worker; this allows us to log an exception (with callstack) as a compilation error and continue the batch
- remove SEH code from ShaderConductorContext, the above makes this redundant (and it didn't provide any actionable information)
- strip down SEH code in D3DShaderCompiler; now only used for the purposes of pre-compiling with DXC in the case of an FXC crash. dumping preprocessed source will be handled in a different manner in a forthcoming CL
- minor change in the DXC precompile path to not log an unnecessary warning when performing an explicitly-requested DXC precompile
#rb Jason.Nadro
#rb Laura.Hermanns
#rb Yuriy.ODonnell
#preflight 63514c798176062ea73acb41
#jira FORT-524383
[CL 22654436 by dan elksnitis in ue5-main branch]
