#ROBOMERGE-SOURCE: CL 16900162 via CL 16917756
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)
[CL 16917910 by krzysztof narkowicz in ue5-release-engine-test branch]
* Generate surfels directly from the triangles instead of relying on ray tracing, to fix coverage issues on some meshes
* Surfels which are inside meshes (surrounded by back faces) or are too close to geometry are discarded
* New surfel clustering algorithm, which inserts one seed after another and tries to iteratively grow clusters in order to find the best set of seeds. Final step is to reset all clusters and grow all simultaneously from previously selected seeds.
* Cluster growing is based on normal, distance and surfel visibility (don�t cluster surfels near geometry first, as it can cause algorithm to be stuck in a local minimum)
* Runtime sampling has strict culling based on the angle and card AABB. Additionally, the tri-planar blending zone was tightened. This improves performance. 0.62-0.5 ms.
* Added various visualizations and CVars for card generation debugging
* Fixed card visibility bug, where card could influence outside of it�s range due to negative shadow map visibility
[FYI] Daniel.Wright, Patrick.Kelly
#ROBOMERGE-SOURCE: CL 16897632 via CL 16916104
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)
[CL 16916378 by krzysztof narkowicz in ue5-release-engine-test branch]
Workaround embree reading 4 bytes off the end of the shared vertex array.
#b Juan.Canada
#rnx
#ROBOMERGE-SOURCE: CL 16673542 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)
[CL 16673546 by matt peters in ue5-release-engine-test branch]
Tested TopazEntry PIE and cooked win64
#ROBOMERGE-SOURCE: CL 15865752 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v786-15839533)
[CL 15865757 by daniel wright in ue5-main branch]
#ROBOMERGE-SOURCE: CL 15828365 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v785-15821902)
[CL 15834014 by daniel wright in ue5-main branch]
#ROBOMERGE-SOURCE: CL 15788010 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v783-15756269)
[CL 15790801 by daniel wright in ue5-main branch]
#ROBOMERGE-SOURCE: CL 15785161 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v783-15756269)
[CL 15790694 by daniel wright in ue5-main branch]
* SDFs are now generated, allocated from the atlas and uploaded in 8^3 bricks (7^3 unique data, half voxel padding).
* Tracing must load the brick index from the indirection table, and only bricks near the surface are stored
* 3 mips are now generated, with the lowest resolution always loaded and the other 2 streamed
* SDFs are now G8 narrow band. Lower resolution mips must be traversed when querying distance to nearest surface far away from the surface
* The Distance Field Brick Atlas is now stored for each FScene and dynamically resized based on needs with a GPU memcopy
* Brick atlas uses a 1d pooled allocator which has no fragmentation and greatly reduces packing waste over the 3d allocator
* Added new indirection for Distance Field Asset data, so that only a single entry needs to be updated when a mip is streamed in or out in scenes with millions of instances
* Compute shaders operating on distance field instances generate streaming requests, which are async read back to CPU, turned into IO requests, which are polled and when complete uploaded to atlases
* Any mesh instance inside the Global SDF extent (200m) requests mip1, and at 50m requests mip2
* Now using a batched compute scatter to upload to the distance field atlas instead of RHIUpdateTexture3d, to bypass alignment restrictions and per-upload overhead
* Distance Field streaming uses an async task to move Memcpy and IO request overhead off of the Rendering Thread
* Distance Field Visualization now computes a normal from the SDF gradient and does simple lighting to better visualize the scene representation
* Increased r.DistanceFields.MaxPerMeshResolution from 128 to 512, to better represent large objects
* Mesh SDF generation now uses an Embree point query to calculate closest unsigned distance, and then a much smaller set of rays to count backfaces for negative region determination, for a 11x speedup
* Upgraded mesh utilities to Embree 3.12.2 to get point queries
* Fixed wrong transform used for SDF normals in Lumen, causing non-uniformly scaled meshes to have incorrect Surface Cache interpolation
* Fixed Static Mesh materials not getting PostLoaded before SDF build, causing their blend modes to be wrong for the build, which corrupts the DDC. Also included those blend modes in the DDC key.
Original costs on 1080 GTX (full updates on everything and no screen traces)
10.60ms UpdateGlobalDistanceField
3.62ms LumenReflectiveTest.DirectionalLight_1 Shadowmap 1
1.73ms VoxelizeCards Clipmaps=[0,1,2,3]
0.38ms TraceCards 1 dispatch 1 groups
0.51ms TraceCards 1 dispatch 1 groups
Sparse SDF costs
12.06ms UpdateGlobalDistanceField
4.35ms LumenReflectiveTest.DirectionalLight_1 Shadowmap 1
2.30ms VoxelizeCards Clipmaps=[0,1,2,3]
0.69ms TraceCards 1 dispatch 1 groups
0.77ms TraceCards 1 dispatch 1 groups
Tested: TopazEntry PC, Reverb PC and PS5, EngineTests, QAGame, Rift, Frosty P_Construct_WP, FortGPUTestbed
#rb Krzysztof.Narkowicz
#ROBOMERGE-OWNER: Daniel.Wright
#ROBOMERGE-AUTHOR: daniel.wright
#ROBOMERGE-SOURCE: CL 15784493 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v783-15756269)
#ROBOMERGE-CONFLICT from-shelf
[CL 15790658 by Daniel Wright in ue5-main branch]
* Moved LumenScene functionality into new LumenScene.cpp file
* Moved all Lumen data into FLumenPrimitive. FPrimitiveSceneInfo now contains just a LumenPrimitiveIndex
* Mesh cards are spawned only near the camera on demand. This saves ~200-300mb of GPU memory on big city levels, and also skips some GPU and CPU costs
* Fixed card merging (previously it was reusing MeshCards which could result in missing cards)
#ROBOMERGE-SOURCE: CL 15494971 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)
[CL 15494982 by krzysztof narkowicz in ue5-main branch]
Phase 1 adds the async compilation without introducing any higher-level awareness.
This step provides the biggest performance improvements by allowing skeletal mesh to be compiled in
multiple threads but stalls are expected to occur before the level has finished loading as soon as
properties of skeletal meshes are accessed by the game-thread.
- Make skeletal mesh reduction and renderdata building thread-safe where needed
- Add protection on all USkeletalMesh accessors so the game-thread can wait on data still being built if needed
- Modify both PostLoad and Build to be async when the feature is enabled
- Make FRawSkeletalMeshBulkData::LoadRawMesh thread-safe
- Fix some PropertyChangedEvent name comparison to use strongly typed GET_MEMBER_NAME_CHECKED
- Add an experimental setting options to enable the feature (disabled by default for now)
DEBUGGING
- Can be forcibly enabled/disabled through command-line via -asyncskeletalmeshcompilation=[off, on, paused]
- Can pause skeletal mesh compilation using Editor.AsyncSkeletalMeshCompilation = 2 or -asyncskeletalmeshcompilation=paused
- Can manually resume a specified amount of paused compilation using Editor.AsyncSkeletalMeshCompilationResume [Num]
- Can forcibly wait on all compilation using Editor.AsyncSkeletalMeshCompilationFlushAll
BENCHMARKS
Tested on AMD TR 3970X with 256GB RAM
- 6m50s to 2m58s for loading P_Construct with a local-only DDC that doesn't contains any prebuilt skeletal mesh
- 3m10s to 39s for importing 299 SK with the Interchange framework
TESTS
- Opened all 5769 SK from FortniteGame to exercise all legacy conversion code paths
- Opened all 102 SK from QAGame
- Opened P_Construct, Apollo_Terrain, LumenTest, P_World
- Imported 299 SK with and without Interchange activated
- Ensure no new regression introduced in any of the 70 SK related tests from EngineTest
- Run all FBX import test in EngineTest
- Cook / Run ShooterGame Client
#rb Alexis.Matte
[CL 15452895 by danny couture in ue5-main branch]
