37 Commits

Author SHA1 Message Date
graham wihlidal
1aec7d15e9 Significant DDC memory and perfomance optimizations to Nanite landscape mesh builds (and some general optimizations to all Nanite mesh builds). In some content cooks with extremely dense Nanite landscape mesh where the process memory well exceeded ~22gb, this brings it down to ~13gb, along with faster processing times.
Highlights:

 * Only allocate and use a wedge map for non-Nanite LODs, or if the mesh is sub 300kindices and it will be cache optimized
 * Only allocate and use a remap if there are no overlapping corners
 * When the Nanite landscape mesh is created, we now ensure the mesh description is fully compacted, computed, and optimized prior to committing it, along with making sure there are no overlapping corners.
 * Since Nanite landscape is checked to ensure no overlapping corners before the build, we can also now skip computing the overlapping corners, which was an intense number of tiny allocations.
 * When Nanite landscape is being built as a static mesh, we now avoid super expensive temporary copies of the mesh description, knowing that no actual mutation will occur.
 * The original code (after building) would release the memory from the mesh description copy, but it would not release the original immutable cached copy from memory (now that is released too)
 * Fixed NaniteLODIndex incorrectly creating a mesh description at a LOD other than the expected 0
 * Removed a full loop over all vertex positions by calculating the bounding box while we're building the vertex buffer (and already processing every vertex)
 * Added a NeedsCompact() helper to MeshDescription to avoid expensive and unnecessary Compact() calls that do lots of reallocs/copies even if the mesh description is already contiguous\packed in memory.
 * Skip building distance field and mesh card representations for the Nanite landscape mesh, which also avoids more cached mesh description deserialization and copies into memory.

Note: In the near future the APIs around this wil be cleaned up further.

#rb richard.talbotwatkin, yuriy.odonnell
[FYI] zousar.shaker, brian.karis, rune.stubbe, juan.canada, danny.couture
#preflight 640b7390482188d710464d39

[CL 24601442 by graham wihlidal in ue5-main branch]
2023-03-10 20:03:17 -05:00
dan thompson
ee896287b8 Warn when building a static mesh that has NaNs in it.
#rb charles.bloom
#preflight 63e6a891c23d1f29a3fbb411

[CL 24137697 by dan thompson in ue5-main branch]
2023-02-11 12:51:08 -05:00
rune stubbe
7b990d37d4 Nanite tangent fixes
-Nanite-specific settings for tangent and degenerates are no longer also applied to the non-nanite mesh LODs on the same mesh.
-100% fallback meshes now have identical tangents to input mesh. Don't skip the tangent generation in StaticMeshBuilder for these meshes and don't recalculate tangents in Nanite builder.

#preflight 632b04c0b40000c8f0c29adb

#rb brian.karis
[FYI] jamie.hayes, graham.wihlidal
#lockdown marc.audy

[CL 22201439 by rune stubbe in ue5-main branch]
2022-09-27 02:14:48 -04:00
rune stubbe
bd86bf40d8 Fix for potential DDC corruption issue where meshes with NaniteEnabled set generate incorrect normals/tangets in projects that have Nanite disabled.
Tangent generation now also respects the Nanite project setting, not just the per-mesh setting.
#lockdown marc.audy

[CL 21862271 by rune stubbe in ue5-main branch]
2022-09-07 17:01:12 -04:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00
Andrew Davidson
3ddc3a4da3 Merge up from //UE5/Dev-LargeWorldCoordinates
#rb none

[CL 16211417 by Andrew Davidson in ue5-main branch]
2021-05-05 15:07:25 -04:00
aurel cordonnier
50944fd712 Merge UE5/RES @ 16162155 to UE5/Main
This represents UE4/Main @ 16130047 and Dev-PerfTest @ 16126156

[CL 16163576 by aurel cordonnier in ue5-main branch]
2021-04-29 19:32:06 -04:00
richard talbotwatkin
b4d72dd359 When importing / building Nanite static meshes, always disable removal of degenerate tris. For clarity, this option is also disabled in the import UI and the Build Settings UI in the Static Mesh Editor, when Nanite is enabled.
#jira UE-108752
#lockdown Simon.Tourangeau
#rb Brian.Karis, Alexis.Matte

#ROBOMERGE-SOURCE: CL 15992767 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v789-15992632)

[CL 15992775 by richard talbotwatkin in ue5-main branch]
2021-04-13 14:54:07 -04:00
Krzysztof Narkowicz
3da3a205fb Added FMeshBuilder::BuildMeshVertexPositions. It replaces FMeshBuilder::BuildMesh(bool bBuildOnlyPosition = true) and does the minimal work which is required to extract vertex positions from a mesh descriptor.
This is later used to build static mesh derived data in cases where Nanite coarse mesh representation replaces original static mesh data and we can't depend on it. In extreme cases it lowers main thread derived data build time from ~7.5s to ~0.1s.

#rb Richard.TalbotWatkin

[CL 14578414 by Krzysztof Narkowicz in ue5-main branch]
2020-10-26 13:09:34 -04:00
Marc Audy
4c1bb11c29 Merge UE5/Release-Engine-Staging to UE5/Main @ 14548662
This represents UE4/Main @ 14525125 + cherrypicked fixes
#skipundocheck

[CL 14551026 by Marc Audy in ue5-main branch]
2020-10-22 19:19:16 -04:00
Krzysztof Narkowicz
5581ebef92 Generate mesh distance fields and Lumen card representation for Nanite meshes directly from the source mesh data. Nanite replaces static mesh data with a coarse mesh representation, which sometimes has holes in it or is not accurate enough. Building from the source data works around that issue and gets up the highest possible SDF/card quality. Unfortunately it also regresses static mesh import times by ~3-5% until we don't get async static mesh builds.
Mesh build numbers for Nanite meshes:
1701216 tris:
Static mesh build: 11.2s
Static mesh build for the SDF async task (added in this CL): 2.3s
Async SDF build: 0.7s
Async card rep build: 0.15s

5524435 tris:
Static mesh build: 36.4s
Static mesh build for the SDF async task: 6.2s
Async SDF build: 1.2s
Async card rep build: 0.5s

1146238 tris
Static mesh build: 10.5s
Static mesh build for the SDF async task: 1.5s
Async SDF build: 0.6s
Async card rep build: 0.1s

#rb Daniel.Wright

[CL 14372405 by Krzysztof Narkowicz in ue5-main branch]
2020-09-22 23:04:05 -04:00
Richard TalbotWatkin
51d4cb3f66 Fully deprecated old MeshDescription APIs.
Prepared direct attribute access for deprecation, preferring use of APIs to access static mesh attributes.
Fixed recently merged Enterprise code to comply with new APIs.
Changed all tools to use triangle-centric iteration where possible.
Added new MeshAttributeArray APIs for querying attribute flags, and added a new Mandatory flag.
Various bug fixes.
#rb Alexis.Matte

[CL 13873755 by Richard TalbotWatkin in ue5-main branch]
2020-07-16 08:23:15 -04:00
Michal Valient
95d19f95b1 [REVERB] Merging //UE4/Private-Reverb-Development@13832732
#rb graham.wihlidal, rune.stubbe, brian.karis, andrew.lauritzen, jeff.farris

[CL 13834854 by Michal Valient in ue5-main branch]
2020-07-06 18:58:26 -04:00
Richard TalbotWatkin
4884bf9d9a First pass of MeshDescription API and format refactor.
- Removed hardcoded element type arrays (Vertices, Edges, Triangles etc.). Mesh element types can now be arbitrarily added, with any number of channels.
- Mesh element containers have a much leaner format; instead of sparse arrays, they are now represented by a simple bitarray, determining whether an index is used or not. Consequently, mesh topology is now entirely described with the attribute system, e.g. edge start and end vertices, triangle vertices, etc.
- Support added for attributes of arbitrary dimensions, e.g. float[4] or int[2].
- Support added for attributes which index into another mesh element container.
- Added FMeshElementIndexer: this is an efficient container for maintaining backward references from one element type to another; for example, edges have an attribute specifying which vertices are at each end (an attribute of type FVertexID[2]). With an indexer, it is possible to look up which edges contain a given vertex, even though this is not explicitly stored. Indexers are designed to do minimal allocations and update lazily and in batch when necessary.
- Added support for preserving UV topology in static meshes. UVs are now a first-class element type which may be indexed directly from triangles.
- Added the facility to access the underlying array in an attribute array directly.
- Triangles now directly reference their vertex, edge and UV IDs. Vertex instances are to be deprecated.
- Changed various systems to be triangle-centric rather than polygon-centric, as this is faster. Triangles are presumed to be the elementary face type in a MeshDescription, even if polygons are still supported. The concept of polygons will be somewhat shifted to mean a group of triangles which should be treated collectively for editing purposes.
- Optimised normal/tangent generation and FBX import.
- Deprecated EditableMesh, MeshEditor and StaticMeshEditorExtension plugins - these are to be removed, but they still have certain hooks in place which need removing.
#rb

[CL 13568702 by Richard TalbotWatkin in ue5-main branch]
2020-05-28 10:56:57 -04:00
JeanMichel Dignard
84facd6d15 Copy from dev-enterprise cl 11097196
#rb none
#rnx

[CL 11099277 by JeanMichel Dignard in Dev-Tools-Staging branch]
2020-01-23 16:28:59 -05:00
ryan durand
471d972e62 Updating copyright for Engine Developer.
#rnx
#rb none


#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)

[CL 10870584 by ryan durand in Main branch]
2019-12-26 15:32:37 -05:00
danny couture
f618e01190 Allow creation of RenderMeshDescription directly from BulkData/RawMesh when MeshDescription is not in memory
Clear MeshDescriptions sooner to reduce memory usage on datasmith import
   - 37.92 GB -> 26.70 GB Peak Working Set when importing DATASET-0008a
   - 66.04 GB -> 52.33 GB Peak Working Set when importing DATASET-0022a

#jira UE-64172
#rb Richard.Talbot-Watkins

#ROBOMERGE-SOURCE: CL 9755311 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v545-9751379)

[CL 9755316 by danny couture in Main branch]
2019-10-23 06:44:22 -04:00
JeanMichel Dignard
d4f0f4d3f2 Copying //UE4/Dev-Enterprise @ cl 9420543 to Dev-Main (//UE4/Dev-Main)
#rb none

[CL 9420574 by JeanMichel Dignard in Main branch]
2019-10-04 13:11:45 -04:00
Chris Gagnon
8ab0638182 Copying //UE4/Dev-Editor to Dev-Main (//UE4/Dev-Main) for 4.24
#rb none

[CL 9325047 by Chris Gagnon in Main branch]
2019-10-01 20:41:42 -04:00
Chris Gagnon
2e87118a18 Copying //UE4/Dev-Editor to Dev-Main (//UE4/Dev-Main) Interim 4.24.
#rb none

[CL 8614014 by Chris Gagnon in Main branch]
2019-09-10 11:35:20 -04:00
Chris Gagnon
8fc25ea18e Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4676797 by Chris Gagnon in Dev-Editor branch]
2019-01-02 14:54:39 -05:00
Ben Marsh
7598af0532 Update copyright notices to 2019.
#rb none
#lockdown Nick.Penwarden

[CL 4662404 by Ben Marsh in Main branch]
2018-12-14 13:41:00 -05:00
Matt Kuhlenschmidt
cde8160913 Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4572510 by Matt Kuhlenschmidt in Dev-Editor branch]
2018-11-16 11:15:08 -05:00
Ben Marsh
ea340d3fbf Copying //UE4/Fortnite-Staging to Dev-Main (//UE4/Dev-Main)
#rb none
#lockdown Nick.Penwarden

[CL 4567513 by Ben Marsh in Main branch]
2018-11-14 19:05:13 -05:00