- Previously the order of shaders would be defined by the order of adding them, which can vary even without any change due to SCWs finishing the jobs at a different time.
- Fixed a previous compile issue and added comments re: TSortedMap and TMap compatibility after checking with the Core team.
#rb Daniel.Lamb (discussed)
[CODEREVIEW] Dmitriy.Dyomin
#ROBOMERGE-SOURCE: CL 11248093 via CL 11248095 via CL 11248101
#ROBOMERGE-BOT: (v647-11244347)
[CL 11254818 by arciel rekman in Main branch]
[FYI] Arciel.Rekman
#ROBOMERGE-SOURCE: CL 11225960 via CL 11228025 via CL 11228838
#ROBOMERGE-BOT: (v644-11213502)
[CL 11241725 by bob tellez in Main branch]
- Previously the order of shaders would be defined by the order of adding them, which can vary even without any change due to SCWs finishing the jobs at a different time.
#rb Daniel.Lamb (discussed)
[CODEREVIEW] Dmitriy.Dyomin
#ROBOMERGE-SOURCE: CL 11224248 via CL 11227707 via CL 11228378
#ROBOMERGE-BOT: (v644-11213502)
[CL 11241606 by arciel rekman in Main branch]
- Optimize parsing of FVertexElement, FDepthStencilStateInitializerRHI, FBlendStateInitializerRHI, FPipelineFileCacheRasterizerState, FPipelineCacheFileFormatPSO using FStringView and TArray<FStringView, TInlineAllocator<N>> to avoid temporary allocations.
- Optimize parsing and string conversion of FCompactFullName, FStableShaderKeyAndValue using FStringView and T[Ansi]StringBuilder to avoid temporary allocations.
- Use TInlineAllocator<16> for FCompactFullName to eliminate dynamic allocations. The size was selected by profiling a title with many shaders.
- Serialize FEditorShaderStableInfo directly to UTF-8 instead of converting to it later to avoid converting every name from ASCII to UTF-16 to UTF-8 during the serialization process.
- Load input files in parallel now that millions of allocations have been eliminated.
- Make the PSOSC serialization self-test only run when verbose logging is enabled.
- Optimize ParseQuoteComma by using FStringView and TArray<FStringView, TInlineAllocator<N>> to avoid temporary allocations.
These changes reduced the execution time of the build command by 90% on a large title. The time savings affect full, iterative, and single-package cooks.
#rb Arciel Rekman, Zousar.Shaker
#ushell-cherrypick of 10860101 by devin.doucette
#ROBOMERGE-SOURCE: CL 11163162 via CL 11163163 via CL 11163164
#ROBOMERGE-BOT: (v640-11091645)
[CL 11166858 by devin doucette in Main branch]
#rnx
#rb none
#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869210 via CL 10869511 via CL 10869900
#ROBOMERGE-BOT: (v613-10869866)
[CL 10870549 by ryan durand in Main branch]
- Added shader pipeline hash as a key attribute.
- Added shaders from shader pipelines (they can have different code if RHI removes unused interpolators between the stages, like D3D does).
#rb Dmitriy.Dyomin, Richard.Wallis
#ROBOMERGE-SOURCE: CL 9959336 via CL 9959339 via CL 9959342
#ROBOMERGE-BOT: (v558-9892490)
[CL 9959347 by arciel rekman in Main branch]
[REVIEW] Ben.Salem, Richard.Wallis
#ROBOMERGE-SOURCE: CL 9868558 via CL 9868562 via CL 9868563
#ROBOMERGE-BOT: (v548-9842178)
[CL 9868569 by chris bunner in Main branch]
Plus - Fix rare shutdown check() failure when async file reads are still wainting in the FShaderPipelineCache.
Plus - Put back correct order of shutdown for FShaderCodeLibrary and FShaderPipelineCache as they can have outstanding aync file handles and we'll get a crash if they are closed after the module shutdown has deleted the FPakPrecacher singleton.
#jira UE-79315
#rb Dmitriy.Dyomin, Michael.Trepka
[FYI] Michael.Trepka
#fyi Richard.Wallis
#ROBOMERGE-OWNER: rolando.caloca
#ROBOMERGE-AUTHOR: richard.wallis
#ROBOMERGE-SOURCE: CL 9743659 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v542-9736015)
[CL 9744080 by rolando caloca in Main branch]
Shader code library was always emitting "<shadermodel>.scl.csv" files while cooking. This output data is only needed for users (or automated jobs) that update the PSO cache. With this change the configuration value "NeedsShaderStableKeys" needs to be set to true for platforms that use PSO (e.g. AndroidEngine.ini and IOSEngine.ini).
This saves diskspace and around 90 seconds on a iterative cook on a large project.
#rb Dmitriy.Dyomin
#ROBOMERGE-SOURCE: CL 6604300 via CL 6604799
#ROBOMERGE-BOT: (v351-6581450)
[CL 6604833 by johan berg in Main branch]
Setting is unchanged for now
[FYI] gil.gribb
#ROBOMERGE-SOURCE: CL 6417507 via CL 6418864 via CL 6418876 via CL 6418960
[CL 6418983 by ben woodhouse in Main branch]
New AssetRegistryState::InitializeFromExistingAndPrune temporarily disabled until some bugs have been fixed.
Test Scenario:
1) BuildCookStageAndRun with these arguments: -platform=Win64 -configuration=Development
2) CookIterate with these arguments: -run=Cook -CookCultures=en -TargetPlatform=WindowsClient -unversioned -stdout -unattended -iterate
Wall Time Results (as an average of running step 2) two times):
Before: ~09:40 (580 seconds) cook commandlet time
After: ~02:30 (150 seconds) cook commandlet time
=> 07:10 (430 seconds ) faster, i.e. a ~ 3.9x speedup
Win32 FileSystem Results:
Before: 1.5 million GetFileAttribute calls and 1.2 million FindNextFile calls
After: 35 0000 GetFileAttribute calls and 1.6 million FindNextFile calls
=> ~400 000 calls to FindNextFile replaces ~1.5 million calls to GetFileAttribute
#rb none
(peafour-cherrypick of //UE4/Dev-Core/[at]5645695 by PJ.Kack)
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 5533504 via CL 5533655 via CL 5536177 via CL 5772728 via CL 5772753
[CL 5772793 by pj kack in Main branch]
