GIsCriticalError was not set to true inside exception handling block, preventing logs in SCW to be dumped from memory to disk despite -LogToMemory argument.
#rnx
#rb sebastian.schoner
[CL 35461655 by laura hermanns in ue5-main branch]
Split Work Graph shaders into multiple frequencies.
This is somewhat in anticipation of graphic nodes.
But also it is a replacement for using CFLAG_WorkgraphLocalNodes to differentiate nodes with local or global root signature.
#rb Yuriy.ODonnell
[CL 33146442 by jeremy moore in ue5-main branch]
- strip old job cache path (constructing an input hash based on all inputs); the cache key based on preprocessed source is now the One True Job Cache
- strip parallelfor-based job submission path (gamethread-blocking); the task path has been enabled for a long while now without issues
- remove the "compile job inputs" debug dump; this is no longer relevant since it's based on inputs to the now-stripped compile job path (similar functionality will be provided by the new form of debug usf once completed)
#rb Laura.Hermanns
[CL 32825320 by dan elksnitis in ue5-main branch]
This will provide the detail needed to connect failed compile jobs (due terminated parent processes) to their UBA job:
Previously, such error looked like this:
> LogUbaController: Display: [CookWorker 4]: LogWindows: Error: appError called: Assertion failed: IFileManager::Get().FileSize(*FilePath) == INDEX_NONE [File:[...]\ShaderCompileWorker.cpp] [Line: 877]
> LogUbaController: Display: [CookWorker 4]: Exiting due to OpenProcess(ParentProcessId) failing and the input file is present!
And now provides the file path:
> LogUbaController: Display: [CookWorker 4]: LogWindows: Error: appError called: Assertion failed: IFileManager::Get().FileSize(*FilePath) == INDEX_NONE [File:[...]\ShaderCompileWorker.cpp] [Line: 877]
> LogUbaController: Display: [CookWorker 4]: Exiting due to OpenProcess(ParentProcessId) failing and the input file is present! FilePath=D:\Example\42.uba.out
#rb henrik.karlsson, Jason.Nadro
#rnx
[CL 31774825 by laura hermanns in ue5-main branch]
It's about 1/4 of CoreMinimal.h but rarely needed (Compression.h pulls on CriticalSection.h and Map.h that are costly).
#rb Yoan.StAmant
[CL 30683417 by aris theophanidis in ue5-main branch]
* Fixed so CrashOutputFile is updated when process is reused (the crash output file should be the same as the output file)
[CL 30643496 by henrik karlsson in ue5-main branch]
Replaced PLATFORM_COMPILER_HAS_DECLTYPE_AUTO blocks.
Replaced PLATFORM_COMPILER_HAS_FOLD_EXPRESSIONS blocks.
[CL 30600164 by steve robb in ue5-main branch]
* Fixed function pointer format for getting next process in uba
* Fixed so output files are written even when return code is non-zero (error)
[CL 30585779 by henrik karlsson in ue5-main branch]
* Removed code checking if worker is running remotely. local and remote should be treated the same way
[CL 30525050 by henrik karlsson in ue5-main branch]
* Changed so when building through uba we don't delete the .in file. Since uba supports rescheduling when remote helpers disconnect we can't delete this file until out is fully sent back
[CL 30468743 by henrik karlsson in ue5-main branch]
- move uniform buffer cleanup and dead stripping into ShaderPreprocessor module's PreprocessShader function
- add "required symbols" to compiler input struct to specify additional symbols to keep during minification aside from those specified by the entrypoint; modify API such that both an entry point string and additional symbols can be specified (to avoid each backend needing to manually parse the compound RT entry point string)
- make use of ModifyShaderCompilerInput in all backends to set additional defines and required symbols on input struct up front; only use the AdditionalDefines map in cases where it's actually necessary
- remove the various per-platform defines for enabling minifier, no longer required now that this has been rolled out for all backends
- fix SCW directcompile mode; this had rotted due to pieces of the FShaderCompilerEnvironment having been added that weren't explicitly serialized to either cmdline or in the shader source. this now serializes as a base64 string written inside the USF containing all portions of the environment required for compilation (using the same serialization function as is used to write/read the SCW input file)
- use a debug flag for indicating we're in "direct compile" mode and should load the debug USF off disk, rather than the poorly named "bSkipPreprocessedCache" (this name is both inaccurate and also confusing due to the addition of the preprocessed job cache)
- modify platform "force wave32" mechanism to use a pragma directive to set a compiler define, instead of doing string replacement in the preprocessed source
- add a view version of the RT entrypoint parsing to use in preprocessing, note that other paths still need to construct fstrings due to further manipulation so keeping the FString path around too
- clean up backends manually checking the "directcompile" cmdline arg
#rb christopher.waters, Yuriy.ODonnell
#rb Chris.Waters
#rb Laura.Hermanns
[CL 30023082 by dan elksnitis in ue5-main branch]
* Inline array memory allocation added to low level preprocessor for output and various temporary buffers to reduce dynamic memory allocation and reallocation overhead. Saved 4.6%.
* FShaderPreprocessOutput::StripCode optimized to write to FString as TCHAR array, rather than using AppendChar (over 4x speedup). Saved 2.9%
* Shader source file cache now also stores stripped and ANSI converted source, to avoid need to convert and strip the source, plus allocating a copy is avoided. Saved 4.3%
* Uniform buffer structure declarations stored as ANSI converted source, avoiding convert and copy. Saved 4.9%
#rnx
#rb dan.elksnitis jason.nadro
[CL 28219741 by jason hoerner in ue5-main branch]
- Remove the condition causing SCW to automatically exit after a single job if HLSLcc was used. This doesn't appear to be beneficial; HLSLcc uses a lot of memory but does not appear to leak it, so we're wasting time spinning up new processes unnecessarily here. This allows us to deprecated the associated field in the compiler output and function in IShaderFormat.
- Strip out compiler invocations stats; this was not accurate in some cases and it was decided after discussion that at this point the effort to fix the problems was not worth the utility we get from it.
#rb Jason.Nadro
#rb Laura.Hermanns
[CL 26256933 by dan elksnitis in ue5-main branch]
