149 Commits

Author SHA1 Message Date
laura hermanns
16602b4bb7 Fix crash on empty string of preprocessed shader source.
#rb Dan.Elksnitis

[CL 26053334 by laura hermanns in 5.3 branch]
2023-06-16 11:21:44 -04:00
dan elksnitis
d688f1fb4d [shaders] debug output improvements/fixes
- remove redundant options struct initialization in DumpDebugShaderData - this was resulting in losing some of the options set in the input parameter options struct
- add explicit "DumpExtendedDebugShaderData" which includes the additional debug output functionality only previously available if an IShaderFormat implementation inherited from FBaseShaderFormat
- add "AdditionalOutputs" functionality which can be used by backends to dump additional (text-based) files following the file naming rules as dictated by the options struct with less boilerplate code
- add a helper for constructing a debug filename from the debug output options struct (some backends require constructing such filenames for i.e. contents of batchfiles; the helper reduces boilerplate code in such cases)
- fix condition for old path of dumping OutputHash.txt to not redundantly dump if shader format supports independent preprocessing and preprocessed cache is enabled

#rb Massimo.Tristano

[CL 25892792 by dan elksnitis in ue5-main branch]
2023-06-09 10:27:24 -04:00
dan elksnitis
06395bee56 [shaders] add Hash field to FShaderCompilerInput so debug output of InputHash.txt works correctly whether or not the preprocessed cache is enabled, and the IShaderFormat API for debug output of independent-preprocessing-enabled backends is cleaner
#rb Jason.Nadro

[CL 25814977 by dan elksnitis in ue5-main branch]
2023-06-06 09:55:43 -04:00
dan elksnitis
2acbcb9533 [shaders]
- add new fields to FShaderCompilerOutput which should be populated with the shader source and entry point name if modified by a backend compile process (in a format implementing the independent preprocessing API). this is intended to supercede "OptionalFinalShaderSource" for such shader formats, as well as being used as the source for preprocessed debug dumps when enabled.
- modify the PCD3D shader format to use these fields as intended (and so fix the debug output for pipelines which remove unused interpolators to match the final modified source).

#rb Laura.Hermanns
#preflight 64777f850848b7126dda194c

[CL 25706448 by dan elksnitis in ue5-main branch]
2023-05-31 13:31:39 -04:00
jason hoerner
88d847a0ad GlobalBeginCompileShader optimizations. 3.2x improvement in performance of the function on Lyra cook:
* Early out when adding uniform buffer resource entries, by checking if uniform buffer had been processed, before processing individual entries.
* Changed resource entry list from a map to an array, saving the cost of map operations adding items to it.
* Resource entries for global uniform buffers cached once at startup.
* Resource entry member names stored in an external buffer, to avoid per-entry string memory allocation overhead.
* Miscellaneous smaller optimizations -- for example, using ByHash to avoid redundant hashing and FString construction.

#jira none
#rnx
#rb jason.nadro dan.elksnitis
#preflight 646cfdbc1134ffac7034af60

[CL 25596308 by jason hoerner in ue5-main branch]
2023-05-23 20:16:11 -04:00
ionut matasaru
5189818fb8 Fixed mismatch string formatting argument in shader compiler (for hlslccflags).
#rb dan.elksnitis
#preflight 64568240fd4b8f4e0d49d747

[CL 25563481 by ionut matasaru in ue5-main branch]
2023-05-22 10:03:04 -04:00
dan elksnitis
f645f93588 [shaders] remove legacy preprocessor option & mcpp library
#rb Jason.Nadro
#preflight 6467bf0c2c0a5da0dcd7aaf2
#fyi Yuriy.ODonnell

[CL 25549427 by dan elksnitis in ue5-main branch]
2023-05-19 14:50:25 -04:00
dan elksnitis
ce8ef1b37c [shaders] fix debug info output when preprocessed job cache is disabled
#preflight 64594f5b9f4768296300db62
#rb Jason.Nadro

[CL 25386339 by dan elksnitis in ue5-main branch]
2023-05-09 08:52:52 -04:00
Steve Robb
0f44db07d5 Replaced some container operator new usage.
#rb james.hopkin
#jira none
#preflight 6459670f28155a0f41565a34

[CL 25384456 by Steve Robb in ue5-main branch]
2023-05-09 05:17:49 -04:00
dan elksnitis
29eff27476 [shaders] build fixes
#preflight none
#rb trivial

[CL 25317313 by dan elksnitis in ue5-main branch]
2023-05-03 10:28:59 -04:00
dan elksnitis
b52faed5d5 [shaders]
- add new IShaderFormat API for separate preprocessing and compilation; backends can implement one or the other depending on the return value of SupportsIndependentPreprocessing
- add support for executing preprocessing in the cook process prior to job submission and constructing job input hashes based on preprocessed source (and a subset of the environment used as compile inputs). controlled by a cvar for now and disabled by default
- add a BaseShaderFormat class in ShaderCompilerCommon which implements common behaviour for output of debug data - note this function is only called for formats which support independent preprocessing, so is expected to be used only by formats which have been converted to use this API
- add new cvars for output of some additional shader debug data - 1. a txt file containing the input hash a.k.a. job cache key 2. a text file containing all diagnostic messages (errors and warnings) for the job
- minor change to how input hashes are constructed for pipeline jobs - sum hashes as 256-bit ints instead of adding to a buffer and re-hashing. faster and simpler, and also more collision resistant (sum of two well distributed hashes equally well distributed)

#rb Jason.Nadro
#rb Yuriy.ODonnell
#preflight 64512c88c86798f650b953d3

[CL 25317218 by dan elksnitis in ue5-main branch]
2023-05-03 10:17:48 -04:00
dan elksnitis
476f6b5335 [shaders]
- add a debug info flags enum/field to indicate which shader debug outputs are desired
- change direct compile txt file mechanism to use this (more options will be added in the near future)
- rename the field on the debug info dump options which indicates that the direct compile file should be skipped to bSourceOnly (when addtional outputs are added these should be skipped as well)

#preflight 6447f6c4b208f61af876e215
#rb Jason.Nadro
#rb Massimo.Tristano

[CL 25183084 by dan elksnitis in ue5-main branch]
2023-04-25 12:45:20 -04:00
dan elksnitis
d934fe3be0 [shaders] move ShaderParameterParser & some related functions to RenderCore so it can be used in the core shader compilation code (forthcoming change requires this)
#preflight 643eda5473470c177c27590d
#rb Yuriy.ODonnell
#rb Christopher.Waters

[CL 25106284 by dan elksnitis in ue5-main branch]
2023-04-19 10:32:12 -04:00
dan elksnitis
5a98357ef7 [shaders] minor change to shader parameter parser parsed param struct to store member data inline rather than a pointer (precursor change to adding serialization to this type)
#preflight 643e9f77a35280ed4f238109
#rb Christopher.Waters

[CL 25085579 by dan elksnitis in ue5-main branch]
2023-04-18 10:06:12 -04:00
christopher waters
ae1bd451f9 Bindless Resources
- Adding correct handling for bindless uniform buffer parameters.
- Reworking RHITextureReference creation to allow the RHI to create them differently, especially with bindless views.
- Adding RHITextureReference bindless view updates to D3D12RHI.

#jira UE-162014
#rb mihnea.balta, jeannoe.morissette
#preflight 642dba03c6769c6082f592f6

[CL 24936647 by christopher waters in ue5-main branch]
2023-04-05 17:03:17 -04:00
Laura Hermanns
cbc4db6343 Minor improvements in shader backends.
#rb Dan.Elksnitis
#jira none
#preflight 642ad56dc6769c6082ade6cf
#rnx

[CL 24892320 by Laura Hermanns in ue5-main branch]
2023-04-03 10:17:45 -04:00
Charles deRousiers
93a0362053 Fix non-determinism in ShaderPrint hashes.
#rb none
#jira UE-181440
#preflight 642472cf803cb466e8c99241
#fyi dan.elksnitis

[CL 24841311 by Charles deRousiers in ue5-main branch]
2023-03-29 14:23:30 -04:00
jeannoe morissette
29f4674898 VulkanRHI: Move to UE::RHICore::SetResourcesFromTables to match other RHIs.
#rb Mihnea.Balta
#preflight 640a3ffed778f88975c84441
#rnx

[CL 24581810 by jeannoe morissette in ue5-main branch]
2023-03-09 15:58:42 -05:00
benjamin rouveyrol
083a8df6f3 StructuredBuffer stride validation: some platforms use the SRV's stride instead of the declared HLSL type to index in structured buffers. This should allow us to catch issues like UE-177214 or UE-178039 by running with -rhivalidation
#rb mihnea.balta
#preflight 6407abb7fb10f393c210ce82
#jira UE-178276
#rnx

[CL 24572578 by benjamin rouveyrol in ue5-main branch]
2023-03-09 03:03:11 -05:00
dan elksnitis
2834fb7c85 [shaders] strip out some 5.2 deprecations
#rb Jason.Nadro
#preflight 63d9227ecf52968117a45f9a

[CL 23926994 by dan elksnitis in ue5-main branch]
2023-01-31 10:14:30 -05:00
Yuriy ODonnell
e5d4d76bed ShaderMinifier - Add basic namespace support
* Track namespaces for symbols
* Re-emit namespace scopes
* Conservatively include symbols from all namespaces in minified code (as if they were all global)
* Additional parsing work will be required to more accurately track used symbols based on active namespaces (i.e. handle `using` keyword, etc.)
* Added RemoveDeadCode() overload that takes an explicit list of required symbols instead of just an entry point name

#rb dan.elksnitis
#preflight 6398be5e35203bc7aa7f9b47

[CL 23494729 by Yuriy ODonnell in ue5-main branch]
2022-12-13 13:42:42 -05:00
yuriy odonnell
a9591ee1e1 ShaderMinifier - Disable some of the debug features by default
#rb none (trivial)
#preflight skip

[CL 23385508 by yuriy odonnell in ue5-main branch]
2022-12-04 02:56:30 -05:00
christopher waters
85bd9bb007 Moving ShaderSymbolExport implementation into its own compilation unit.
#jira none
#rb matt.peters
#preflight 637d24918b12eb83a7ddea74

[CL 23241530 by christopher waters in ue5-main branch]
2022-11-22 16:14:14 -05:00
dan elksnitis
a3253bae46 [shaders] tiny change to allow skipping output of directcompile.txt file ignoring bool in compile input (used in some forthcoming debug code which dumps extra copies of preprocessed shader code and doesn't need these)
#preflight 6377fcab8b12eb83a7e6e3d1
#rb Jason.Nadro

[CL 23235560 by dan elksnitis in ue5-main branch]
2022-11-22 10:25:53 -05:00
matt peters
df72d2d4e9 ShaderCompilerCommon: Fix missing symbols in monolithic editor build due to spurious inline keyword in on function definition.
#rb None, trivial
#rnx
#preflight None, fixing build break

[CL 23212668 by matt peters in ue5-main branch]
2022-11-18 20:36:49 -05:00