danny couture ff158b6e95 Async Texture Compilation ...

- Feature can be activated in the Experimental section of the Editor Settings
 - Replace Texture2D/TextureCube resources by placeholders until their PlatformData is ready
 - Add a utility class allowing to encapsulate raw field pointers without breaking compatibility
 - Protect PlatformData from unsafe access through encapsulation.
 - Protect texture's resource from race conditions between game and render threads through encapsulation.
    - This allows to get rid of FlushRenderingCommands and long game-thread stutters when Updating a texture's resource.
    - UpdateResource was never safe to call without a FlushRenderingCommands and multiple call-site are doing exactly that, this will fix those cases.
    - Those were probably undetected due to their low occurence rate under normal conditions but can easily be reproed during async texture compilation on 32 cores.
 - Force wait on required texture compilations for MaterialBaking, ProxyMesh, Thumbnail generation for disk usage
 - Wait on all textures compilation whenever a wait for all shaders compilation is requested for safety (i.e. screenshot)
 - Compile UI and heightmap textures with higher priority to reduce visual artefacts
 - Increase priority of texture that have been rendered to improve time-to-usefulness of the editor under low core count
 - Async compilation is disabled for -game / non-editor mode as there is currently no support for async bulk data loading from external files
 - Properly cancel async tasks when UTexture is garbage collected before the compilation is finished
 - Show progress when explicitly waiting on compilation
 - Changing the mip settings in the texture editor (or any settings requiring the running platform data to be recomputed) will now be processed asynchronously.

DEBUGGING

 - Can be forcibly enabled/disabled through command-line via -asynctexturecompilation=[off, on, paused]
 - Can pause texture compilation using Editor.AsyncTextureCompilation = 2 or -asynctexturecompilation=paused
 - Can manually resume a specified amount of paused compilation using Editor.AsyncTextureCompilationResume [Num]
 - Can forcibly wait on all compilation using Editor.AsyncTextureCompilationFlushAll

BENCHMARKS

 - 3m15s to 1m20s when loading Apollo_Terrain with no textures in DDC (AMD TR 3970X)
 - 6m45s to 1m11s when loading Apollo_Terrain with no textures in DDC (-corelimit=8)
 - 3m10s to 1m54s when lauching PIE on Apollo_Terrain with no textures in DDC (AMD TR 3970X)
 - 7m43s to 1m36s when lauching PIE on Apollo_Terrain with no textures in DDC (-corelimit=8)
 - 0m57s to 0m42s when importing Attic_NVIDIA.usd with no textures in DDC (AMD TR 3970X)
 - 2m14s to 0m35s when importing Attic_NVIDIA.usd with no textures in DDC (-corelimit=4)

TESTS
 - Success on all material baking tests from EngineTests with -asynctexturecompilation=paused
 - Runned with -corelimit=1 all the way to unlimited
 - Cooking worked
 - Opening the texture editor/material editor will force the compilation to finish like expected.
 - Changing a setting in the texture editor will recompile async, even allowing to close the editor and continue doing other changes.
 - Unpausing the compilation will update the texture thumbnails properly.
 - Started with -asynctexturecompilation=paused, and then unpaused after a map loading, and then into a PIE session to stresstest UpdateResources.
 - Tested both dx11/dx12
 - Vulkan fails on Fortnite even with -asynctexturecompilation=off because of Landscape weigthmap, not this CL.
 - Compiled and tested FortniteGame / UE4 / ShooterGame projects

#rb Uriel.Doyon, Francis.Hurteau

[CL 13694814 by danny couture in ue5-main branch]

2020-06-16 22:16:25 -04:00

..

Private

Async Texture Compilation

2020-06-16 22:16:25 -04:00

Public

Copying //UE4/Dev-RenderPlat-Staging@11388153 to //UE4/Main

2020-02-12 13:27:19 -05:00

MaterialUtilities.Build.cs

Copy from dev-enterprise cl 11097196

2020-01-23 16:28:59 -05:00