219 Commits

Author SHA1 Message Date
matthew sorrels
0dc0af5b5f Fix crash when cooking Android OpenGLES on Linux by replacing swscanf based parser with native FindNextHLSLDefinitionOfType
#jira UE-205727
#rnx
#rb dan.elksnitis, Dmitriy.Dyomin, Florin.Pascu

[CL 32500348 by matthew sorrels in 5.4 branch]
2024-03-26 03:00:44 -04:00
carl lloyd
6c66d3e9f3 Fix for incorrect regex expression when renaming uniform buffers in OpenGL
#rb tim.doerries

[CL 30880732 by carl lloyd in 5.4 branch]
2024-01-25 05:14:54 -05:00
dan elksnitis
a9037b25ed [shaders]
- add FShaderSource class which wraps source as populated by preprocessing and subsequently accessed by compilation and other debug features; this class automatically inserts zeroed padding such that 16-byte-wide SIMD string comparison operations do not require a non-SIMD tail to process any overhang.
- add typedefs for the string/view/character types and update preprocessing code to use these typedefs instead of the explicit types
- add explicit if constexprs in minifier code around char width to disable simd optimizations for char width != 2 (and subsequently skip the non-simd tail if char width == 2 since FShaderSource automatically adds the required padding)

#rb Jason.Nadro, Yuriy.ODonnell

[CL 30358137 by dan elksnitis in ue5-main branch]
2023-12-15 15:28:27 -05:00
dan elksnitis
cc7c2c54f4 [shaders] shader format preprocessing cleanup & refactoring
- move uniform buffer cleanup and dead stripping into ShaderPreprocessor module's PreprocessShader function
- add "required symbols" to compiler input struct to specify additional symbols to keep during minification aside from those specified by the entrypoint; modify API such that both an entry point string and additional symbols can be specified (to avoid each backend needing to manually parse the compound RT entry point string)
- make use of ModifyShaderCompilerInput in all backends to set additional defines and required symbols on input struct up front; only use the AdditionalDefines map in cases where it's actually necessary
- remove the various per-platform defines for enabling minifier, no longer required now that this has been rolled out for all backends
- fix SCW directcompile mode; this had rotted due to pieces of the FShaderCompilerEnvironment having been added that weren't explicitly serialized to either cmdline or in the shader source. this now serializes as a base64 string written inside the USF containing all portions of the environment required for compilation (using the same serialization function as is used to write/read the SCW input file)
- use a debug flag for indicating we're in "direct compile" mode and should load the debug USF off disk, rather than the poorly named "bSkipPreprocessedCache" (this name is both inaccurate and also confusing due to the addition of the preprocessed job cache)
- modify platform "force wave32" mechanism to use a pragma directive to set a compiler define, instead of doing string replacement in the preprocessed source
- add a view version of the RT entrypoint parsing to use in preprocessing, note that other paths still need to construct fstrings due to further manipulation so keeping the FString path around too
- clean up backends manually checking the "directcompile" cmdline arg

#rb christopher.waters, Yuriy.ODonnell
#rb Chris.Waters
#rb Laura.Hermanns

[CL 30023082 by dan elksnitis in ue5-main branch]
2023-11-30 15:56:34 -05:00
christopher waters
6f3f63b186 Bindless Conversion Changes
- Removing the need for COMPILER_DEFINE on GetResourceFromHeap and GetSamplerFromHeap
- FShaderParameterParser constructor now requires a FPlatformConfiguration object that each shader format can configure.
- FPlatformConfiguration::GenerateBindlessAccess allows each platform to generate the code for accessing a bindless resource or sampler.
- Added EShaderParameterParserConfigurationFlags to specify support for stable constant buffers and bindless.
- Moved EBindlessParameterMode into EShaderParameterParserConfigurationFlags::BindlessUsesArrays
- While moving things around, I was able to convert some TCHAR* uses to FStringView

#jira UE-166341
#rb mihnea.balta

[CL 29533376 by christopher waters in ue5-main branch]
2023-11-07 16:26:20 -05:00
florian penzkofer
27b98dc65a Fix crash in OpenGL shader compiler on macOS
#ushell-cherrypick of 29304271 by florian.penzkofer
[FYI] dmytro.ivanov
#rb dan.elksnitis

[CL 29420329 by florian penzkofer in ue5-main branch]
2023-11-03 16:33:03 -04:00
florian penzkofer
cd93ccdb8b Support ThinTranslucent Materials in mobile renderer
OpenGL ES and Metal use framebuffer fetch.
Vulkan uses dual source blending.
For Vulkan and OpenGL ES there is a fallback shader permutation for drivers that don't support this. The fallback is the same as the existing solution that uses regular blending (i.e. looks different).
Others uses dual source blending and we force use of DXC for those shaders.
#rb Dmitriy.Dyomin, Florin.Pascu

[CL 29245271 by florian penzkofer in ue5-main branch]
2023-10-30 15:41:46 -04:00
christopher waters
5c304ed095 Removing FShaderParameterParser from FShaderPreprocessOutput.
#jira UE-197328
#rb dan.elksnitis

[CL 29168356 by christopher waters in ue5-main branch]
2023-10-26 23:07:19 -04:00
dan elksnitis
54fb5cf56d [shaders] further deprecation (SupportsIndependentPreprocessing/OptionalFinalShaderSource/bIndependentPreprocessed are no longer needed since all backends are migrated)
#rb Laura.Hermanns

[CL 29117027 by dan elksnitis in ue5-main branch]
2023-10-25 22:21:30 -04:00
jason hoerner
0e18ea7e81 Shader Preprocessor: Roll TEXT macro substitution into the C preprocessor step, via custom macro callback feature, making it effectively free. Saves 3% overall on preprocess, but also represents a simplification by reducing the number of unique preprocess passes.
#jira UE-198496
#rnx
#rb dan.elksnitis yuriy.odonnell jason.nadro

[CL 29100104 by jason hoerner in ue5-main branch]
2023-10-25 16:01:11 -04:00
christopher waters
6797229f23 Moving more platforms to use ShaderParameterParser in the compile step.
#rb Jason.Nadro
#jira UE-197328

[CL 28655234 by christopher waters in ue5-main branch]
2023-10-11 03:20:39 -04:00
christopher waters
bb0a274b8c Reworking UniformBuffer code generation to not use an intermediate struct
- When generating Uniform Buffer HLSL code, a 'static const struct {} Name' was created which has been causing compiler issues off and on for years.
- When RemoveUniformBuffersFromSource removed said struct, it parsed the struct's structure to figure out how to translate UniformBuffer.Member into UniformBuffer_Member in code that accessed Uniform Buffers.
- Now, instead of a struct, a "UniformBuffer <Name>" declaration is created and filled with mappings from "UniformBuffer.Member" into the correct code.
- This allows ConstantBuffer objects to be used differently from cbuffer where the former generates "UniformBuffer.Member" and the latter generates "UniformBuffer_Member"
- This also allows resources to always be translated to "UniformBuffer_Member"
- Shader define 'PLATFORM_SUPPORTS_UB_STRUCT' is now 'PLATFORM_SUPPORTS_CONSTANTBUFFER_OBJECT' and will be set on shader formats that support ConstantBuffer<> objects.
- Removed lots of unused code from BindlessResources.ush

#jira UE-195103
#rb dan.elksnitis, jeannoe.morissette

[CL 28058373 by christopher waters in ue5-main branch]
2023-09-20 17:33:36 -04:00
christopher waters
1a3a1d4e90 Changing DumpDebugInfoPath conditions to use a helper function.
#rb dan.elksnitis

[CL 27681962 by christopher waters in ue5-main branch]
2023-09-07 14:45:23 -04:00
dan elksnitis
88169dbd8c [shaders] implement independent preprocessing IShaderFormat API for OpenGL shader format
#rb Christopher.Waters
#rb Massimo.Tristano

[CL 27482746 by dan elksnitis in ue5-main branch]
2023-08-30 09:38:41 -04:00
dan elksnitis
d0128a9fa0 [shaders]
- enable minifier deadstripping for OpenGL shader format; bytecode is a 100% match with this change on multiple tested projects so it seems safe to enable
- deprecate FOpenGLFrontend; nothing in available code appears to inherit from it anymore so the API appears unnecessary. this makes the OpenGL shader format more similar to others (which do not expose compilation functions as API)

#rb Jason.Nadro
#rb Laura.Hermanns
#rb Yuriy.ODonnell

[CL 27338803 by dan elksnitis in ue5-main branch]
2023-08-24 09:09:10 -04:00
david harvey
da9b5b86f5 Do not include PreWindowsApi.h / PostWindowsApi.h directly in code.
- mostly just replacing them AllowWindowsPlatformTypes.h / HideWindowsPlatformTypes.h
 - some other files had both Pre/PostWindowsApi.h and Allow/HideWindowsPlatformTypes.h, so just removed
 - some occasional include order changes.

#jira UE-152863
#rnx
#rb Devin.Doucette

[CL 26985385 by david harvey in ue5-main branch]
2023-08-10 03:34:53 -04:00
david harvey
bdb4199eeb Remove unnecessary WindowsHWrapper.h & MinWindows.h include - both files will be automatically included by AllowWindowsPlatformTypes.h
#jira UE-152863
#rnx
#rb Josh.Adams

[CL 26912096 by david harvey in ue5-main branch]
2023-08-08 05:50:53 -04:00
dan elksnitis
9a5a1b6f74 [shaders] minor include cleanup
#rb Jason.Hoerner
#rb Laura.Hermanns

[CL 26817718 by dan elksnitis in ue5-main branch]
2023-08-03 13:40:32 -04:00
dan elksnitis
5f85b46590 [shaders] remove unnecessary passing of hlslcc flags as an option to debug info output. this was just used to pass a switch on the SCW directcompile cmdline which SCW no longer does anything with. also cleaned up some additional unnecessary code related to these flags in backends that no longer use hlslcc at all.
#rb Laura.Hermanns

[CL 26685354 by dan elksnitis in ue5-main branch]
2023-07-28 15:58:01 -04:00
jason hoerner
7f22080814 Shader Compiler: Made FShaderCompilerDefinitions private, to reduce code publicly visible in ShaderCore.h. For now, it's just marked deprecated for 5.4, but will be hidden more generally in 5.5.
#rnx
#rb yuriy.odonnell dan.elksnitis jason.nadro

[CL 26534998 by jason hoerner in ue5-main branch]
2023-07-22 06:35:10 -04:00
florin pascu
b5a6b18005 Fix GenerateGlslShader when a string is a substring of another in the ReflectData
#jira UE-185872
#preflight 646338a7cd6910375684092d

[CL 25494932 by florin pascu in ue5-main branch]
2023-05-16 15:28:39 -04:00
Steve Robb
0f44db07d5 Replaced some container operator new usage.
#rb james.hopkin
#jira none
#preflight 6459670f28155a0f41565a34

[CL 25384456 by Steve Robb in ue5-main branch]
2023-05-09 05:17:49 -04:00
jeannoe morissette
efa00a9cb5 OpenGL: Avoid StorageImageWriteWithoutFormat SPIRV capability when compiling shaders for GL by adding a "-fvk_force_storage_image_format" flag that brings back old behavior (promote 3-component UAV to 4-component instead of using Unknown format). (author: Carl.Lloyd)
Broken since 24910389.

#rb Florin.Pascu,Carl.Lloyd
#fyi Laura.Hermanns,Dmitriy.Dyomin
#preflight 6442c73d0171aae0b1c6e85e
#jira UE-183868,UE-183432
#rnx

[CL 25149146 by jeannoe morissette in ue5-main branch]
2023-04-21 13:52:26 -04:00
dan elksnitis
26464b581b [shaders] further rework of parameter parser API to simplify the independent preprocessing use case, plus adding serialization support and some minor cleanup.
#preflight 6442af0ec023bb1f734fc0d1
#rb Christopher.Waters

[CL 25148947 by dan elksnitis in ue5-main branch]
2023-04-21 13:45:43 -04:00
dan elksnitis
49d7985a85 [shaders] Add "compile argument" functionality for generic parameters consumed by compile backends, along with helper to set both a compile arg AND define for cases which require both. The purpose of this is to move towards enforcing that preprocessor defines should _only_ be used during the preprocessing phase itself; the subset of defines that are currently used only as arguments to the compile backends (or used as both actual preprocessor defines and backend arguments) should use this mechanism instead. This allows us to explicitly identify and hash such parameters to be included in a stable key based primarily on a hash of preprocessed/deadstripped source; without it we would need to hash all defines which will defeat most of the deduplication benefit of doing this. A subsequent change will deprecate the GetDefinitions function on the shader compilation environment to enforce that defines cannot be read by anything outside of core shader system functionality.
#rb Massimo.Tristano
#rb Laura.Hermanns
#preflight 64258c2e5e52099fe3d03625

[CL 24854110 by dan elksnitis in ue5-main branch]
2023-03-30 10:47:09 -04:00