Tested TopazEntry PIE and cooked win64
#ROBOMERGE-SOURCE: CL 15865752 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v786-15839533)
[CL 15865757 by daniel wright in ue5-main branch]
#ROBOMERGE-SOURCE: CL 15828365 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v785-15821902)
[CL 15834014 by daniel wright in ue5-main branch]
#ROBOMERGE-SOURCE: CL 15788010 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v783-15756269)
[CL 15790801 by daniel wright in ue5-main branch]
#ROBOMERGE-SOURCE: CL 15785161 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v783-15756269)
[CL 15790694 by daniel wright in ue5-main branch]
* SDFs are now generated, allocated from the atlas and uploaded in 8^3 bricks (7^3 unique data, half voxel padding).
* Tracing must load the brick index from the indirection table, and only bricks near the surface are stored
* 3 mips are now generated, with the lowest resolution always loaded and the other 2 streamed
* SDFs are now G8 narrow band. Lower resolution mips must be traversed when querying distance to nearest surface far away from the surface
* The Distance Field Brick Atlas is now stored for each FScene and dynamically resized based on needs with a GPU memcopy
* Brick atlas uses a 1d pooled allocator which has no fragmentation and greatly reduces packing waste over the 3d allocator
* Added new indirection for Distance Field Asset data, so that only a single entry needs to be updated when a mip is streamed in or out in scenes with millions of instances
* Compute shaders operating on distance field instances generate streaming requests, which are async read back to CPU, turned into IO requests, which are polled and when complete uploaded to atlases
* Any mesh instance inside the Global SDF extent (200m) requests mip1, and at 50m requests mip2
* Now using a batched compute scatter to upload to the distance field atlas instead of RHIUpdateTexture3d, to bypass alignment restrictions and per-upload overhead
* Distance Field streaming uses an async task to move Memcpy and IO request overhead off of the Rendering Thread
* Distance Field Visualization now computes a normal from the SDF gradient and does simple lighting to better visualize the scene representation
* Increased r.DistanceFields.MaxPerMeshResolution from 128 to 512, to better represent large objects
* Mesh SDF generation now uses an Embree point query to calculate closest unsigned distance, and then a much smaller set of rays to count backfaces for negative region determination, for a 11x speedup
* Upgraded mesh utilities to Embree 3.12.2 to get point queries
* Fixed wrong transform used for SDF normals in Lumen, causing non-uniformly scaled meshes to have incorrect Surface Cache interpolation
* Fixed Static Mesh materials not getting PostLoaded before SDF build, causing their blend modes to be wrong for the build, which corrupts the DDC. Also included those blend modes in the DDC key.
Original costs on 1080 GTX (full updates on everything and no screen traces)
10.60ms UpdateGlobalDistanceField
3.62ms LumenReflectiveTest.DirectionalLight_1 Shadowmap 1
1.73ms VoxelizeCards Clipmaps=[0,1,2,3]
0.38ms TraceCards 1 dispatch 1 groups
0.51ms TraceCards 1 dispatch 1 groups
Sparse SDF costs
12.06ms UpdateGlobalDistanceField
4.35ms LumenReflectiveTest.DirectionalLight_1 Shadowmap 1
2.30ms VoxelizeCards Clipmaps=[0,1,2,3]
0.69ms TraceCards 1 dispatch 1 groups
0.77ms TraceCards 1 dispatch 1 groups
Tested: TopazEntry PC, Reverb PC and PS5, EngineTests, QAGame, Rift, Frosty P_Construct_WP, FortGPUTestbed
#rb Krzysztof.Narkowicz
#ROBOMERGE-OWNER: Daniel.Wright
#ROBOMERGE-AUTHOR: daniel.wright
#ROBOMERGE-SOURCE: CL 15784493 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v783-15756269)
#ROBOMERGE-CONFLICT from-shelf
[CL 15790658 by Daniel Wright in ue5-main branch]
* Moved LumenScene functionality into new LumenScene.cpp file
* Moved all Lumen data into FLumenPrimitive. FPrimitiveSceneInfo now contains just a LumenPrimitiveIndex
* Mesh cards are spawned only near the camera on demand. This saves ~200-300mb of GPU memory on big city levels, and also skips some GPU and CPU costs
* Fixed card merging (previously it was reusing MeshCards which could result in missing cards)
#ROBOMERGE-SOURCE: CL 15494971 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)
[CL 15494982 by krzysztof narkowicz in ue5-main branch]
Phase 1 adds the async compilation without introducing any higher-level awareness.
This step provides the biggest performance improvements by allowing skeletal mesh to be compiled in
multiple threads but stalls are expected to occur before the level has finished loading as soon as
properties of skeletal meshes are accessed by the game-thread.
- Make skeletal mesh reduction and renderdata building thread-safe where needed
- Add protection on all USkeletalMesh accessors so the game-thread can wait on data still being built if needed
- Modify both PostLoad and Build to be async when the feature is enabled
- Make FRawSkeletalMeshBulkData::LoadRawMesh thread-safe
- Fix some PropertyChangedEvent name comparison to use strongly typed GET_MEMBER_NAME_CHECKED
- Add an experimental setting options to enable the feature (disabled by default for now)
DEBUGGING
- Can be forcibly enabled/disabled through command-line via -asyncskeletalmeshcompilation=[off, on, paused]
- Can pause skeletal mesh compilation using Editor.AsyncSkeletalMeshCompilation = 2 or -asyncskeletalmeshcompilation=paused
- Can manually resume a specified amount of paused compilation using Editor.AsyncSkeletalMeshCompilationResume [Num]
- Can forcibly wait on all compilation using Editor.AsyncSkeletalMeshCompilationFlushAll
BENCHMARKS
Tested on AMD TR 3970X with 256GB RAM
- 6m50s to 2m58s for loading P_Construct with a local-only DDC that doesn't contains any prebuilt skeletal mesh
- 3m10s to 39s for importing 299 SK with the Interchange framework
TESTS
- Opened all 5769 SK from FortniteGame to exercise all legacy conversion code paths
- Opened all 102 SK from QAGame
- Opened P_Construct, Apollo_Terrain, LumenTest, P_World
- Imported 299 SK with and without Interchange activated
- Ensure no new regression introduced in any of the 70 SK related tests from EngineTest
- Run all FBX import test in EngineTest
- Cook / Run ShooterGame Client
#rb Alexis.Matte
[CL 15452895 by danny couture in ue5-main branch]
- Moved source models related classes and implementation into separate files.
- Created a new UObject wrapper for holding mesh description bulk data; this is created as a transactable object inside a UStaticMesh.
- The cached mesh description is now an inner object of the bulk data wrapper; this allows transactions on the bulk data to affect the cached mesh.
- The static mesh SourceModels array can no longer be accessed directly: FStaticMeshSourceModel requires extra initialization before it should be used, so new LODs should be added through the static mesh API.
- Undo/redo is now fixed within the geometry tools
- Fixed regression where mesh description was being unpacked unnecessarily during transactions
- MeshDescription bulk data and its cached MeshDescriptions are now emptied prior to removing the UObject reference, in order to immediately free memory instead of needing to wait for garbage collection
- No more deadlocks when clearing or committing mesh descriptions during garbage collection
- Bulk data UObjects are constructed, where possible, at startup, in order to prevent it from happening during critical moments, e.g. during garbage collection or package saving.
#rb Danny.Couture, Alexis.Matte, Ryan.Schmidt
[CL 15420827 by Richard TalbotWatkin in ue5-main branch]
i.e. This allows latency sensitive texture compression parallelfor from the gamethread to benefit from all the foreground threads and complete faster
#rb Arne.Schober
[CL 15387719 by danny couture in ue5-main branch]
* r.LumenScene.MeshCardsMaxLOD controls representation quality. With 0 being a box projection
* New card placement to focus on important parts of a mesh (skip non visible parts)
* Replaced CubeMapTrees with MeshCards. Instead of a big volume texture lookup there are now 6 lists of decals (one per axis)
* Merged distance field and mesh card Embree setup code into a set of common functions
* Removed hardcoded limits of 6 cards per mesh
#fyi Daniel.Wright, Patrick.Kelly, Yuriy.ODonnell
[CL 15144947 by Krzysztof Narkowicz in ue5-main branch]
