Commit Graph

85 Commits

Author SHA1 Message Date
dan elksnitis
cc7c2c54f4 [shaders] shader format preprocessing cleanup & refactoring
- move uniform buffer cleanup and dead stripping into ShaderPreprocessor module's PreprocessShader function
- add "required symbols" to compiler input struct to specify additional symbols to keep during minification aside from those specified by the entrypoint; modify API such that both an entry point string and additional symbols can be specified (to avoid each backend needing to manually parse the compound RT entry point string)
- make use of ModifyShaderCompilerInput in all backends to set additional defines and required symbols on input struct up front; only use the AdditionalDefines map in cases where it's actually necessary
- remove the various per-platform defines for enabling minifier, no longer required now that this has been rolled out for all backends
- fix SCW directcompile mode; this had rotted due to pieces of the FShaderCompilerEnvironment having been added that weren't explicitly serialized to either cmdline or in the shader source. this now serializes as a base64 string written inside the USF containing all portions of the environment required for compilation (using the same serialization function as is used to write/read the SCW input file)
- use a debug flag for indicating we're in "direct compile" mode and should load the debug USF off disk, rather than the poorly named "bSkipPreprocessedCache" (this name is both inaccurate and also confusing due to the addition of the preprocessed job cache)
- modify platform "force wave32" mechanism to use a pragma directive to set a compiler define, instead of doing string replacement in the preprocessed source
- add a view version of the RT entrypoint parsing to use in preprocessing, note that other paths still need to construct fstrings due to further manipulation so keeping the FString path around too
- clean up backends manually checking the "directcompile" cmdline arg

#rb christopher.waters, Yuriy.ODonnell
#rb Chris.Waters
#rb Laura.Hermanns

[CL 30023082 by dan elksnitis in ue5-main branch]
2023-11-30 15:56:34 -05:00
wouter dek
ee216585d8 Fix HLSLcc compilation on MacOS and Linux and recompile binaries.
- the xcode and cmake projects were updated to use cpp17
- a variable was renamed in opt_copy_propagation_elements to fix a compilation error on linux
- remove the register keyword from the autogenerated lexer code using a define
- added and modified scripts to fix the compilation process

#rb Laura.Hermanns

[CL 29925957 by wouter dek in ue5-main branch]
2023-11-25 06:43:38 -05:00
dan elksnitis
54fb5cf56d [shaders] further deprecation (SupportsIndependentPreprocessing/OptionalFinalShaderSource/bIndependentPreprocessed are no longer needed since all backends are migrated)
#rb Laura.Hermanns

[CL 29117027 by dan elksnitis in ue5-main branch]
2023-10-25 22:21:30 -04:00
stu mckenna
c051ba798e - Fix NaN's poisoning the VVM constant table
- Comparing A==B will return true for NaN depending on the FP mode, so we now compare if both values are finite before comparing for equality.
- This does not fix the source of the error, but will stop all subsequent float values being set to NaN

#rb rob.krajcarski
#jira UE-194597

[CL 27694043 by stu mckenna in ue5-main branch]
2023-09-07 18:51:56 -04:00
dan elksnitis
9064172949 [shaders] migration of VectorVM shader format to independent preprocessing API (along with some cleanup)
#rb Rob.Krajcarski
#rb Laura.Hermanns

[CL 27106719 by dan elksnitis in ue5-main branch]
2023-08-15 12:28:03 -04:00
dan elksnitis
5f85b46590 [shaders] remove unnecessary passing of hlslcc flags as an option to debug info output. this was just used to pass a switch on the SCW directcompile cmdline which SCW no longer does anything with. also cleaned up some additional unnecessary code related to these flags in backends that no longer use hlslcc at all.
#rb Laura.Hermanns

[CL 26685354 by dan elksnitis in ue5-main branch]
2023-07-28 15:58:01 -04:00
jason hoerner
7f22080814 Shader Compiler: Made FShaderCompilerDefinitions private, to reduce code publicly visible in ShaderCore.h. For now, it's just marked deprecated for 5.4, but will be hidden more generally in 5.5.
#rnx
#rb yuriy.odonnell dan.elksnitis jason.nadro

[CL 26534998 by jason hoerner in ue5-main branch]
2023-07-22 06:35:10 -04:00
dan elksnitis
5c00384f67 [shaders]
- Remove the condition causing SCW to automatically exit after a single job if HLSLcc was used. This doesn't appear to be beneficial; HLSLcc uses a lot of memory but does not appear to leak it, so we're wasting time spinning up new processes unnecessarily here. This allows us to deprecated the associated field in the compiler output and function in IShaderFormat.
- Strip out compiler invocations stats; this was not accurate in some cases and it was decided after discussion that at this point the effort to fix the problems was not worth the utility we get from it.

#rb Jason.Nadro
#rb Laura.Hermanns

[CL 26256933 by dan elksnitis in ue5-main branch]
2023-06-27 10:11:41 -04:00
kirill zorin
b6ee3a6c64 Fix UE_LOG callsites that have format string-related UB
#rb trivial
#preflight 647107780a6634dbb236a0a6

[CL 25651638 by kirill zorin in ue5-main branch]
2023-05-26 15:52:39 -04:00
stu mckenna
fd6cae6b42 - Add VVM support for bit casting between floats and ints
#rb rob.krajcarski
#preflight 6466532e743f7c995bc67a0f

[CL 25548196 by stu mckenna in ue5-main branch]
2023-05-19 13:51:29 -04:00
bryan sefcik
da92084a12 Optimized out more private modules includes and dependencies.
#preflight 64627c382965f6ea8ea83bd6

[CL 25479683 by bryan sefcik in ue5-main branch]
2023-05-15 16:26:12 -04:00
dan elksnitis
34c034f2b2 [shaders] add a mutex acquired around all calls into hlslcc as a temporary fix for parallel in-process compilation. also some minor cleanup (include sorting, remove an unused/unimplemented function prototype)
#rb Jason.Nadro
#rb Yuriy.ODonnell
#rb Laura.Hermanns
#preflight 63e272b08374ef68dfd142ea

[CL 24049810 by dan elksnitis in ue5-main branch]
2023-02-07 11:04:07 -05:00
dan elksnitis
ecab3b4a9a [shaders] refactor usf/directcompile.txt dump into a utility in ShaderCompilerCommon to reduce code duplication in the shader backends
#rb Jason.Nadro
#rb Christopher.Waters
#preflight 635bf1e1df01edd149f5727e

[CL 22829548 by dan elksnitis in ue5-main branch]
2022-10-28 11:45:49 -04:00
rob krajcarski
a5489c3279 Fix for mistaken static analysis warning with ir_vm generation
#preflight none
#rb none

[CL 22175566 by rob krajcarski in ue5-main branch]
2022-09-24 20:52:54 -04:00
rob krajcarski
5e02132d25 Fixes to niagara VM bytecode generation for if blocks where a variable is assigned and then used within the same block
if (Condition)
{
  Foo = Bar;	// this would have been replaced with:	branch_flattened_temp0 = Bar;
  Foo2 = Foo;	// this would have been replaced with:	branch_flattened_temp1 = Foo; this should have been replaced by:
				branch_flattened_temp1 = branch_flattened_temp0;
}
#jira UE-164327
#rb simon.tovey
#preflight 632ddb75e23e50651b95cca4

[CL 22164425 by rob krajcarski in ue5-main branch]
2022-09-23 20:23:13 -04:00
bryan sefcik
de1956f47b Ran IWYU on Public headers under Engine/Source/Developer/...
Headers are updated to contain any missing #includes needed to compile and #includes are sorted.  Nothing is removed.

#ushell-cherrypick of 21064294 by bryan.sefcik
#jira
#preflight 62d5c2111062f2e63015e598

#ROBOMERGE-OWNER: bryan.sefcik
#ROBOMERGE-AUTHOR: bryan.sefcik
#ROBOMERGE-SOURCE: CL 21155249 via CL 21158121 via CL 21161259
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21182053 by bryan sefcik in ue5-main branch]
2022-07-20 12:03:45 -04:00
Robb Surridge
0a5a02c5b2 Coding standard fixes: gender-inclusive language
#jira UE-156429
#preflight 62b32e826a25ba6ae52f94bd
#rb jason.walter

[CL 20795375 by Robb Surridge in ue5-main branch]
2022-06-23 11:14:07 -04:00
rob krajcarski
c18bc543e3 Fix for VM bytecode generation which was failing in a very small number of cases of the form:
A = GetFirstValue();
B = A;
A = GetSecondValue();
C = A; // <-- this would get GetSecondValue() assigned to C instead of GetFirstValue()

Adds an SSA pass to help reduce overwriting registers, adds logic to remove self assignments, and adds logic to stop propagating when values are dirtied

From a sampling of scripts, about 2% are changed (where 1-3 unnecessary ops have been deleted).

#rb simon.tovey
#preflight 62b0d63c8e03312ad83a0c51
#jira UE-148892

[CL 20756957 by rob krajcarski in ue5-main branch]
2022-06-21 09:55:03 -04:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
69fe095547 Merge from Release-Engine-Staging @ 17636544 to Release-Engine-Test
This represents UE4/Main @17638339 and Dev-PerfTest @17636504

[CL 17638842 by aurel cordonnier in ue5-release-engine-test branch]
2021-09-27 19:54:25 -04:00
aurel cordonnier
02c0f425e8 Copy up from Release-Engine-Staging @ 16738359
This represents UE4/Main @ 16738161 and Dev-PerfTest @ 16737719

[CL 16738582 by aurel cordonnier in ue5-release-engine-test branch]
2021-06-22 00:27:54 -04:00
Marc Audy
e80ea6b959 Merge from Release-Engine-Staging @ 16444985
This represents UE4/Main @ 16445039 and Dev-PerfTest @ 16444526

[CL 16445122 by Marc Audy in ue5-release-engine-test branch]
2021-05-25 02:43:26 -04:00
Simon Tovey
c4d171f9a6 Fix for VM compile issue with matrix constants.
#rb Rob.Krajcarski
#jira UE-115957, UE-115956

[CL 16377212 by Simon Tovey in ue5-main branch]
2021-05-18 17:59:25 -04:00
Marc Audy
0cbbc781ca Merge UE5/Release-Engine-Staging @ 15740152 to UE5/Main
This represents UE4/Main @ 15709114

[CL 15740605 by Marc Audy in ue5-main branch]
2021-03-18 15:20:03 -04:00
Marc Audy
50a3d7d368 Merge Release-Engine-Staging to Main @ CL# 14467590
This represents UE4/Main @ 14432125 + some cherrypick fixes

[CL 14468207 by Marc Audy in ue5-main branch]
2020-10-09 22:42:26 -04:00