* Added GroupIdenticalMeshesForBaking, which, when activated, will result in identical meshes (or mesh instances) being baked only once.
Other changes/fixes
* Always rely on the StaticMesh RenderData when retrieving mesh description for mesh merging/simplification (was the case when retrieving from a SMC, now also the case for a SM).
* For some reason, Material baking was rendering each triangle twice, but with opposite facing. Uneeded as we are rendering with backface culling disabled.
[CL 15669055 by Sebastien Lussier in ue5-main branch]
- Feature can be activated in the Experimental section of the Editor Settings
- Replace Texture2D/TextureCube resources by placeholders until their PlatformData is ready
- Add a utility class allowing to encapsulate raw field pointers without breaking compatibility
- Protect PlatformData from unsafe access through encapsulation.
- Protect texture's resource from race conditions between game and render threads through encapsulation.
- This allows to get rid of FlushRenderingCommands and long game-thread stutters when Updating a texture's resource.
- UpdateResource was never safe to call without a FlushRenderingCommands and multiple call-site are doing exactly that, this will fix those cases.
- Those were probably undetected due to their low occurence rate under normal conditions but can easily be reproed during async texture compilation on 32 cores.
- Force wait on required texture compilations for MaterialBaking, ProxyMesh, Thumbnail generation for disk usage
- Wait on all textures compilation whenever a wait for all shaders compilation is requested for safety (i.e. screenshot)
- Compile UI and heightmap textures with higher priority to reduce visual artefacts
- Increase priority of texture that have been rendered to improve time-to-usefulness of the editor under low core count
- Async compilation is disabled for -game / non-editor mode as there is currently no support for async bulk data loading from external files
- Properly cancel async tasks when UTexture is garbage collected before the compilation is finished
- Show progress when explicitly waiting on compilation
- Changing the mip settings in the texture editor (or any settings requiring the running platform data to be recomputed) will now be processed asynchronously.
DEBUGGING
- Can be forcibly enabled/disabled through command-line via -asynctexturecompilation=[off, on, paused]
- Can pause texture compilation using Editor.AsyncTextureCompilation = 2 or -asynctexturecompilation=paused
- Can manually resume a specified amount of paused compilation using Editor.AsyncTextureCompilationResume [Num]
- Can forcibly wait on all compilation using Editor.AsyncTextureCompilationFlushAll
BENCHMARKS
- 3m15s to 1m20s when loading Apollo_Terrain with no textures in DDC (AMD TR 3970X)
- 6m45s to 1m11s when loading Apollo_Terrain with no textures in DDC (-corelimit=8)
- 3m10s to 1m54s when lauching PIE on Apollo_Terrain with no textures in DDC (AMD TR 3970X)
- 7m43s to 1m36s when lauching PIE on Apollo_Terrain with no textures in DDC (-corelimit=8)
- 0m57s to 0m42s when importing Attic_NVIDIA.usd with no textures in DDC (AMD TR 3970X)
- 2m14s to 0m35s when importing Attic_NVIDIA.usd with no textures in DDC (-corelimit=4)
TESTS
- Success on all material baking tests from EngineTests with -asynctexturecompilation=paused
- Runned with -corelimit=1 all the way to unlimited
- Cooking worked
- Opening the texture editor/material editor will force the compilation to finish like expected.
- Changing a setting in the texture editor will recompile async, even allowing to close the editor and continue doing other changes.
- Unpausing the compilation will update the texture thumbnails properly.
- Started with -asynctexturecompilation=paused, and then unpaused after a map loading, and then into a PIE session to stresstest UpdateResources.
- Tested both dx11/dx12
- Vulkan fails on Fortnite even with -asynctexturecompilation=off because of Landscape weigthmap, not this CL.
- Compiled and tested FortniteGame / UE4 / ShooterGame projects
#rb Uriel.Doyon, Francis.Hurteau
[CL 13694814 by danny couture in ue5-main branch]
CL 10373564 by danny.couture
Optimize Material Baking (Phase 1)
- Introduce a mecanism to override the vertex/index buffer allocator used for dynamic meshes
- Avoid GDynamicMesh non-ticked pools build-up by using our own vertex/index buffer pool during baking
- Reduce reallocation and incurred soft page faults by reusing a single set of vertex/index buffers big enough for the biggest mesh
- Preemptively detect if smearing would result in monochrome texture to avoid useless work
- Shrink smeared monochrome textures during the baking process for huge memory savings
- Move UV smearing in worker threads to avoid blocking the game thread
- Required shaders are now built asynchronously
- Add progress bar for material baking
- 28m23 [at] 150 GB RAM -> 2m14s [at] 45 GB RAM for 6 channels [at] 512x512 when baking materials on ProxyLOD for DATASET-0008a with DDC empty
#rb Jurre.deBaare, Sebastien.Lussier
CL 10516258 by danny.couture
Optimize Material Baking (Phase 2)
- Implement pipelining with staging buffers to avoid GPU stalls when reading from render targets
- Reuse the same prepared FMeshBatch instead of rebuilding it for each draw pass
- Prepare the RenderItem in advance on other threads to reduce work on the game thread
- Move the staging surface copy out of the render thread
- Small vertex and index buffers are not reused to avoid dependency locks when mapping them
- Fix bug in Canvas Flush_RenderThread found while running HLOD rebuild commandlet on Fortnite
- Delete old and unused MaterialBakingModule.h from public files
- 4m44s -> 59s for baking 6 channel [at] 1024x1024 when baking materials on ProxyLOD for DATASET-0008a with shaders already compiled
- Time spent in Material Baking when rebuilding all HLOD on Apollo_POI_Large_HLOD (Phase 1 + 2 combined)
- 10m18s -> 2m36s for a first rebuild all in editor with no shaders in DDC (cold)
- 1m23s -> 20s for a second rebuild all in editor (warm)
#rb Jeremy.Moore, Sebastien.Lussier
CL 11135986 by sebastien.lussier
Optimized mesh merging
* Added DeletePolygons() & DeleteTriangles methods to FMeshDescription which rely on TSets<> instead of performing costly TArray::AddUnique() calls()
* Parallelized UV generation and avoided duplicate processing of the same mesh+lod pairs
* Optimized FMeshDescriptionOperations::GenerateUniqueUVsForStaticMesh()
* Goes from 100s to 10s in my test case
#rb danny.couture, jeanfrancois.dube, richard.talbotwatkin
#ROBOMERGE-OWNER: sebastien.lussier
#ROBOMERGE-AUTHOR: sebastien.lussier
#ROBOMERGE-SOURCE: CL 11206337 via CL 11206341 via CL 11206346
#ROBOMERGE-BOT: (v643-11205221)
[CL 11206493 by sebastien lussier in Main branch]
#rnx
#rb none
#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)
[CL 10870584 by ryan durand in Main branch]
#rb jeanfrancois.dube
#ROBOMERGE-SOURCE: CL 8362258 via CL 8364673 via CL 8368019
#ROBOMERGE-BOT: (v402-8361577)
[CL 8368177 by sebastien lussier in Main branch]
* Fixed stale materials in MaterialProxyPool
* Fixed memory leak
#jira UE-78913
[REVIEW] jeanfrancois.dube
[FYI] bryce.lumpkin
#ROBOMERGE-SOURCE: CL 8126046 via CL 8126053 via CL 8131826
#ROBOMERGE-BOT: (v401-8057353)
[CL 8156535 by sebastien lussier in Main branch]
[FYI] bryce.lumpkin
#ROBOMERGE-SOURCE: CL 7629383 via CL 7629384 via CL 7637104
#ROBOMERGE-BOT: (v372-7473910)
[CL 7637265 by sebastien lussier in Main branch]
#fix Make sure we retain Magenta pixels for emissive baked textures, and not apply emissive scaling to them to make sure they are masked out correctly when merging the different textures into the atlas
#rb none
#ROBOMERGE-SOURCE: CL 6104752 via CL 6104754 via CL 6105018
[CL 6105094 by jurre debaare in Main branch]
