#fyi dan.elksnitis
Original CL Desc
-----------------------------------------------------------------
[shaders] unify handling of single job bSucceeded flag; wasn't getting set properly when not using workers
#rb Yuriy.ODonnell
#rb Laura.Hermanns
#rb Jason.Nadro
#preflight 63d2e011d21dbe1d29be43c1
[CL 23889694 by dan elksnitis in ue5-main branch]
- move compile job types into RenderCore and use them in SCW
- using the above factor out a helper function for pipeline job compilation to further reduce code duplication
- make flags indicating job success/failure conditions part of the SCW output data rather than evaluating them when reading the output (so they can be evaluated once in the helper function mentioned above)
- modify debug worker input dumps to write a single file for combined-pipeline compilation rather than one file per stage
#rb Yuriy.ODonnell
#preflight 63d1725d574ab9cae4c93d76
[CL 23868225 by dan elksnitis in ue5-main branch]
this has much potential for misuse, and is generally incompatible with the goal of in-process multithreaded compilation
#rb Jason.Nadro
#rb Yuriy.ODonnell
#preflight 63c94ba902024f93d83dfbbc
[CL 23772691 by dan elksnitis in ue5-main branch]
[FYI] Laura.Hermanns
Original CL Desc
-----------------------------------------------------------------
Add host machine name to OOM report in DXC backend and refactor global SCW error-code handling.
#rb Jason.Nadro, Dan.Elksnitis, Yuriy.Odonnell, Arciel.Rekman
#preflight 638e610c255f07df8ea212f4
#rnx
[CL 23421737 by kate ellis in ue5-main branch]
- allow 0 TimeToLive to be specified explicitly on commandline (useful when debugging single-job debug worker input files, without this it keeps running the same job repeatedly) and modify single job worker input commandline txt file to use this instead of 0.5f
- change config-conditional behaviour serializing to memory instead of file on disk to be conditional on an optional cmdline argument instead, and move file copy into the block that does the file writing (otherwise it repeatedly tries and fails to copy the nonexistent file)
#preflight 6377fed033774509008c42c6
#rb Jason.Nadro
[CL 23234934 by dan elksnitis in ue5-main branch]
- fix collisions in debug dumps for material shaders - name the folder based on the material instance name (if building a material instance) instead of just the base material name, also hash the asset path and static parameters to differentiate materials with the same name and different path and different static permutations
- add a debug dump option for writing single-job ShaderCompileWorker input files to allow debugging SCW in normal operation mode (including a separate txt file containing commandline args for SCW to do so)
- use path concatenation operator instead of + in shadercompileworker when constructing input/output file paths
#preflight 6362a45d6000216d2e15cd5b
#rb Christopher.Waters
#rb Rolando.Caloca
#rb Yuriy.ODonnell
[CL 22920594 by dan elksnitis in ue5-main branch]
- Focused around moving GlobalBeginCompileShader and friends.
- ModifyCompilationEnvironment and ValidateCompiledResult now only compiled in Editor builds.
- Measured 0.5MB to 1.0MB elf size reduction depending on platform.
#jira none
#rb jason.nadro, arciel.rekman, florin.pascu
#preflight 63613f992b5338aceb442902
[CL 22890964 by christopher waters in ue5-main branch]
- each platform compilation request is now wrapped in __try/__except rather than a single exception handling block at the top level of the worker; this allows us to log an exception (with callstack) as a compilation error and continue the batch
- remove SEH code from ShaderConductorContext, the above makes this redundant (and it didn't provide any actionable information)
- strip down SEH code in D3DShaderCompiler; now only used for the purposes of pre-compiling with DXC in the case of an FXC crash. dumping preprocessed source will be handled in a different manner in a forthcoming CL
- minor change in the DXC precompile path to not log an unnecessary warning when performing an explicitly-requested DXC precompile
#rb Jason.Nadro
#rb Laura.Hermanns
#rb Yuriy.ODonnell
#preflight 63514c798176062ea73acb41
#jira FORT-524383
[CL 22654436 by dan elksnitis in ue5-main branch]
This can be enabled by modifying `UnrealEngine\Engine\Saved\UnrealBuildTool\BuildConfiguration.xml` like so:
<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
<BuildConfiguration>
<bShaderCompilerWorkerTrace>true</bShaderCompilerWorkerTrace>
</BuildConfiguration>
</Configuration>
- Added a build configuration xml value, `bShaderCompilerWorkerTrace`.
- Turning this on will set USE_SHADER_COMPILER_WORKER_TRACE=1
- Move the parameter -nothreading to be set when we launch the process instead of internally as an extra cmd line arg.
- Unreal Insights uses a separate thread to send events so threading support is needed for the program. When we have USE_SHADER_COMPILER_WORKER_TRACE enabled we need to turn off `-nothreading`.
- When USE_SHADER_COMPILER_WORKER_TRACE is enabled we pass in `-trace=default` to get CPU event markers.
- The SCW program needs to turn on the following defines to be able to perform CPU and memory traces:
ENABLE_LOW_LEVEL_MEM_TRACKER=1
UE_MEMORY_TAGS_TRACE_ENABLED=1
UE_TRACE_ENABLED=1
- Instrument Shader Compiler Worker with TRACE_CPUPROFILER_EVENT_SCOPE. This are no-ops when this is turned off.
todo: Make the shader compiler worker inherit the trace args from the main process it was launched from.
#rb Yuriy.ODonnell
#jira none
#preflight 634ef80269246074db9637c2
[CL 22625183 by Jason Nadro in ue5-main branch]
Changed ShaderArchive, GlobalShaderCache, ShaderDebugInfo and Autogen to use ShaderPlatformName and not ShaderFormat when naming their output files.
#rb Jack.Porter, Chris.Waters, Mihnea.Balta, Jason.Nadro
#jira UE-120561
#preflight 62c31f6fc9410537282296c6
[CL 20937870 by Florin Pascu in ue5-main branch]
