Add queued work priority support to auto delete async tasks and schedule DDC Puts as Low priority
ThreadPool threads are now honoring the names they've been given on creation
Move the I/O only async DDC Get into the DDC ThreadPool with higher priority (Normal) than DCC Puts
#rb Devin.Doucette
[CL 15177897 by danny couture in ue5-main branch]
- In the original code we were checking if either of the queues were nullptr before using them however when those queues are deleted they were not actually being set to nullptr, so the checks would not catch this and eventually we would read trash data when trying to use the queues.
- Set both the queues to nullptr on shutdown so no attempt to access them will be made after this point.
#rb Danny.Couture
#rnx
[CL 15066339 by paul chipchase in ue5-main branch]
This avoids a crash when the first use of DDC is from a worker thread when global shader map compilation is skipped due to use of the null RHI.
#jira UE-104924
#rb Zousar.Shaker
[CL 15039561 by Devin Doucette in ue5-main branch]
Create a new interface IPackageResourceManager that handles all attempts to load packages from storage. The default implementation passes through to loading from IFileManager. Future implementations will load from other services such as the EditorDomain and TargetDomain.
Add FPackagePath class that holds a LocalFilePath or LongPackageName, or a relative path to a package under a mounted content root (which can be converted into either the LocalFilePath or LongPackageName).
Modify LinkerLoad, BulkData, and AsyncLoader to use FPackagePath and IResourceManager.
Modify UPackage and FLinkerLoad to have a FPackagePath instead of a Filename.
#rb Paul.Chipchase, CarlMagnus.Nordin, Francis.Hurteau, Devin.Doucette
#rn Major Core
[CL 14814912 by Matt Peters in ue5-main branch]
- Will help to converge a lot faster when thousands of small mesh are part of a scene
- Will also reduce distance field atlas fragmentation due to improved batch ordering
#rb Francis.Hurteau
[CL 14734205 by danny couture in ue5-main branch]
- Add protection against async property accessed during async build and postload.
- Split PostLoad into 3 steps with the most part now being made async.
- Make Build/BatchBuild API async when the feature is activated.
- Add a new StaticMesh compilation manager. (This will be refactory later to reduce duplicated code with texture compiler).
- Skip RenderState creation for any static mesh component still being built so they only show up when ready.
- Fixed DistanceField and MeshCard computation that might trigger too soon if fixups are required to RenderData during PostLoad.
- Dynamic priorisation for building and updating static meshes nearest to the viewport.
- Implement different PIE modes, current default is to stall only for mesh that might affect nagivation/collision for players/bots.
- Add a new generic interface for assets being built (still a WIP).
- Add ability in FrontEnd filters to skip serialization of asset being async compiled and refresh when compilation finishes.
- Prevent auto-save while textures and static meshes are being built (same as shaders)
- Logic has been reordered in some Fortnite Building Component to avoid unnecessarily touching static mesh properties.
General Optimizations
- Use cached thumbnails until shaders/textures/static meshes are ready to improve performance and avoid rendering incomplete thumbnails.
DEBUGGING
- Can be forcibly enabled/disabled through command-line via -asyncstaticmeshcompilation=[off, on, paused]
- Can pause staticmesh compilation using Editor.AsyncStaticMeshCompilation = 2 or -asyncstaticmeshcompilation=paused
- Can manually resume a specified amount of paused compilation using Editor.AsyncStaticMeshCompilationResume [Num]
- Can forcibly wait on all compilation using Editor.AsyncStaticMeshCompilationFlushAll
BENCHMARKS
- 2m19s to 17.9s for Loading FortGPUTestbed's LumenTest with an empty local only DDC
- 2h45m to 5mxxs for Loading Reverb's P_World with an empty local only DDC
- 17m29 to 10m27s for Loading Apollo_Terrain_Edit with an empty local only DDC
TESTS
- Loading maps from Reverb, Fortnite, ShooterGame, QAGame, FortGPUTestbed
- Content browser interactions while still under heavy compilation
- Importing with Datasmith / Dataprep
- Working with the new Mesh Modeling Tool
- Shutting down the editor while loading
- Most of EngineTest Test Suite
REFERENCES
- Design https://docs.google.com/document/d/1O4GI1G9AtQN6l0SaGFfXw0DyNhlLunLoEqne5INvxIQ
- Documentation https://docs.google.com/document/d/1KCdFEMhhcsGwfgOisTVwlOwtsmEd7qbB0V6Tc39Gb10
#rb Francis.Hurteau
[CL 14688146 by danny couture in ue5-main branch]
- Update primitive component's relying on a texture to avoid GPU cached values to become stale (i.e. Lumen, Virtual Textures).
- Reduce flickering during texture update by removing virtual texture flush as it is not required anymore with the previous fix.
- Prioritize on screen textures even for nanite mesh and virtual textures by looking at "last seen" primitive component material's usage.
- Force heighmap texture to be ready for PIE and -game to avoid falling through the landscape.
General Optimizations
- Eliminated FlushRenderingCommands from both MeshCards/DistanceFields update to reduce heavy stuttering.
- Reuse expensive ObjectIterator scans between MeshCards/DistanceFields/Shaders/Textures/StaticMesh during the same tick.
#rb Francis.Hurteau
[CL 14649969 by danny couture in ue5-main branch]
