- Add a new API to release preloaded shader to provide a way to manage preloaded data (which was previously statically allocated).
#rb James.Doverspike, Richard.Wallis
[REVIEW] [at]Allan.Bentham, [at]Richard.Wallis
#preflight 625e11c4db15ac92db7ee858
#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 19832275 via CL 19833640 via CL 19837454 via CL 19854946 via CL 19855339
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v940-19807014)
[CL 19856713 by arciel rekman in ue5-main branch]
- "Build" command of the PSO commandlet will split the cache, generating separate files for each chunk.
- Criterion is whether any shaders of the PSO can be used by the assets in that chunk (including after deduplication).
- Disabled by default because the runtime part to pick up those files isn't in yet.
#rb Allan.Bentham, Dmytro.Vovk
[REVIEW] [at]Allan.Bentham, [at]Pete.Sauerbrei, [at]Chris.Genova, [at]James.Doverspike, [at]Kenzo.TerElst, [at]Dmytro.Vovk
#preflight 62448dc2e434babd8ae73354
#ROBOMERGE-OWNER: arciel.rekman
#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 19563493 via CL 19563649 via CL 19563674
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v937-19513599)
[CL 19565457 by arciel rekman in ue5-main branch]
- Pipeline caches had serialization code for the new types but wasn't creating the shaders on preload
- Fixing the cache commandlet to be flexible with the supported shader types.
#jira none
#rb jason.nadro, arciel.rekman
[CL 15801914 by christopher waters in ue5-main branch]
- Shaders are searched by TypeName and PermutationId, so move those fields out of FShader and into separate arrays inside ShaderMap, to reduce cache misses while finding shaders
#rb none
#ROBOMERGE-OWNER: ben.ingram
#ROBOMERGE-AUTHOR: ben.ingram
#ROBOMERGE-SOURCE: CL 12646854 via CL 12646966 via CL 12646981
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v675-12543919)
[CL 12646995 by ben ingram in Main branch]
- Each shadermap is now associated with one or more assets, and shaders are laid out according to the
open order of those assets, if provided. If not provided, the original ordering is kept. The original ordering
will not be always deterministic, particularly if shader compilation needs to happen, because jobs can get
reordered during the compilation.
Other fixes:
- Fixed sorting within the shadermap not being deterministic (FCompareShaderPrimaryKey)
- Fixed Niagara shadermap duplicating Platform property and not initializing it properly.
- Removed (unused) support for Child shader libraries (would not be deterministic).
#rb Ben.Ingram, Rob.Krajcarski
#ROBOMERGE-OWNER: Arciel.Rekman
#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 12249260 via CL 12249262 via CL 12250173
#ROBOMERGE-BOT: (v668-12245121)
[CL 12250313 by Arciel Rekman in Main branch]
Including StringFwd.h in place of forward declarations offers more flexibility to change how these types are declared.
#jira none
#rb Steve.Robb
[RN] minor Core
#ROBOMERGE-SOURCE: CL 12235779 in //UE4/Release-4.25/... via CL 12235792
#ROBOMERGE-BOT: RELEASE (Release-4.25Plus -> Main) (v664-12234175)
[CL 12237495 by devin doucette in Main branch]
#rb Josh.Adams, Ben.Woodhouse
#rnx
#ROBOMERGE-OWNER: Arciel.Rekman
#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 11450810 via CL 11452365 via CL 11452390 via CL 11452432
#ROBOMERGE-BOT: (v654-11333218)
[CL 11456136 by Arciel Rekman in Main branch]
- Optimize parsing of FVertexElement, FDepthStencilStateInitializerRHI, FBlendStateInitializerRHI, FPipelineFileCacheRasterizerState, FPipelineCacheFileFormatPSO using FStringView and TArray<FStringView, TInlineAllocator<N>> to avoid temporary allocations.
- Optimize parsing and string conversion of FCompactFullName, FStableShaderKeyAndValue using FStringView and T[Ansi]StringBuilder to avoid temporary allocations.
- Use TInlineAllocator<16> for FCompactFullName to eliminate dynamic allocations. The size was selected by profiling a title with many shaders.
- Serialize FEditorShaderStableInfo directly to UTF-8 instead of converting to it later to avoid converting every name from ASCII to UTF-16 to UTF-8 during the serialization process.
- Load input files in parallel now that millions of allocations have been eliminated.
- Make the PSOSC serialization self-test only run when verbose logging is enabled.
- Optimize ParseQuoteComma by using FStringView and TArray<FStringView, TInlineAllocator<N>> to avoid temporary allocations.
These changes reduced the execution time of the build command by 90% on a large title. The time savings affect full, iterative, and single-package cooks.
#rb Arciel Rekman, Zousar.Shaker
#ushell-cherrypick of 10860101 by devin.doucette
#ROBOMERGE-SOURCE: CL 11163162 via CL 11163163 via CL 11163164
#ROBOMERGE-BOT: (v640-11091645)
[CL 11166858 by devin doucette in Main branch]
#rnx
#rb none
#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869210 via CL 10869511 via CL 10869900
#ROBOMERGE-BOT: (v613-10869866)
[CL 10870549 by ryan durand in Main branch]
- Added shader pipeline hash as a key attribute.
- Added shaders from shader pipelines (they can have different code if RHI removes unused interpolators between the stages, like D3D does).
#rb Dmitriy.Dyomin, Richard.Wallis
#ROBOMERGE-SOURCE: CL 9959336 via CL 9959339 via CL 9959342
#ROBOMERGE-BOT: (v558-9892490)
[CL 9959347 by arciel rekman in Main branch]
Shader code library was always emitting "<shadermodel>.scl.csv" files while cooking. This output data is only needed for users (or automated jobs) that update the PSO cache. With this change the configuration value "NeedsShaderStableKeys" needs to be set to true for platforms that use PSO (e.g. AndroidEngine.ini and IOSEngine.ini).
This saves diskspace and around 90 seconds on a iterative cook on a large project.
#rb Dmitriy.Dyomin
#ROBOMERGE-SOURCE: CL 6604300 via CL 6604799
#ROBOMERGE-BOT: (v351-6581450)
[CL 6604833 by johan berg in Main branch]
