tim doerries
118a044620
Fix for jittering SparseVolumeTexture frames when importing a sequence with different translations in the transforms of all the input files/frames. A popular third party tool for baking volumetrics unfortunately always creates such sequences. The UnrealEngine .vdb importer assumes that all frames of a sequence share the same transform and effectively ignores that transform. This fix detects changes in the translation and bakes the translation into the created SVT, eliminating the jittering.
...
#rb Sebastian.Hillaire
#jira UE-194916
#rnx
[CL 27881724 by tim doerries in 5.3 branch]
2023-09-14 12:08:00 -04:00
tim doerries
f055d39e56
Fix for SparseVolumeTexture .vdb importer creating broken assets when importing .vdb sequences with this pattern: FILENAME.NUMBER.vdb. When importing a sequence, the importer creates an animated SVT and uses the filename with the number stripped at the end as asset/package name. With files using this pattern, we end up with a dot at the end of the asset/package name, which violates UE naming rules and makes the asset completely unusable without any proper warnings or error messages. The fix now also strips the dot (and other invalid chars).
...
#rb ?
#jira UE-194914
#rnx
[CL 27754452 by tim doerries in 5.3 branch]
2023-09-11 11:05:40 -04:00
tim doerries
e294c57cda
SparseVolumeTexture import dialog now defaults to 16bit float for all components and tries to use 8bit unorm for density if possible.
...
#rb Sebastien.Hillaire
#rnx
[CL 26706161 by tim doerries in 5.3 branch]
2023-07-31 10:45:05 -04:00
Joe Kirchoff
077153323d
Remove unnecessary Public & Private include paths for subdirectories within a module
...
#rnx
#jira none
[CL 26560072 by Joe Kirchoff in 5.3 branch]
2023-07-24 20:07:27 -04:00
tim doerries
46063588a6
Fixed SparseVolumeTextureSamplePageTable() and SparseVolumeTextureLoadPageTable() sometimes loading from the page table at incorrect coordinates when the page table is using a page table offset. Fixed incorrect page table offset in SVT sequences causing parts of the volume to appear cut off at higher mip levels. Fixed empty vdb frames during import causing massive VolumeResolution values in the resulting SVT. Fixed StreamingManager triggering an assert when all voxels of a higher mip level have been stripped away due to occupancy bitmask based compression. It seems that higher mip levels can sometimes be empty despite lower mip levels having valid data. I suspect that this is due to precision issues when averaging 8 small float values and then quantizing them during the mip mapping.
...
Removed unused lambda in SVT factory.
#rb none
#jira UE-189125
#rnx
[CL 26209573 by tim doerries in 5.3 branch]
2023-06-23 10:14:28 -04:00
tim doerries
089a74e6ef
Implemented automatic asset renaming when importing .vdb sequences as an animated SparseVolumeTexture. The resulting asset name will no longer have the number postfix of the original file used to import the entire sequence.
...
SVT assets now also have an additional property to show the number of frames in the editor UI.
#rb Sebastien.Hillaire
#rnx
[CL 26090848 by tim doerries in 5.3 branch]
2023-06-19 10:56:55 -04:00
tim doerries
6d2b2eeffd
Implemented support for reimporting SparseVolumeTextures. Also fixed some inconsistent dllexport macros which initially caused problems when trying to call generated functions of the SVT classes in the factory (which resides in a different module).
...
#rb Sebastien.Hillaire
#rnx
[CL 26037038 by tim doerries in 5.3 branch]
2023-06-16 01:57:43 -04:00
tim doerries
838de94380
New streaming system for SparseVolumeTextures. It is based on the Nanite FStreamingManager design and supports streaming both from disk and from DDC. This first version uses a single logical tile data texture for all frames of a single UStreamableSparseVolumeTexture and starts streaming out tiles of older mip levels once it runs out of space. The highest mip level (lowest resolution) is always resident in the tile data texture, so there is always something available for rendering. Each frame's page table texture is currently always fully resident. There is now also a buffer storing the lowest resident mip level index for each frame.
...
In a future version, additional logical tile data textures may be allocated to handle cases where more mip levels are being requested than can physically fit into a single texture. In addition, page table textures should also be resized so that streamed out frames take up less GPU memory. It might also be necessary to implement a blocking streaming option for MRQ and similar use cases. This feature probably depends on being able to spill to additional physical page table textures.
Also moved all of the SVT runtime classes into a shared namespace (UE::SVT), which is why this CL ended up touching almost all SVT related files.
#rb Sebastien.Hillaire, Rune.Stubbe, Devon.Penney, Patrick.Kelly
#rnx
#preflight 64772ef20d55081f54759f0b
[CL 25699866 by tim doerries in ue5-main branch]
2023-05-31 08:14:22 -04:00
bryan sefcik
91c57d395e
Removed redundant module includes.
...
#preflight 645d4bf3aa3c584c0b5b3a67
[CL 25435653 by bryan sefcik in ue5-main branch]
2023-05-11 16:48:21 -04:00
tim doerries
fedfdecf2f
Implemented an optimized path for creating a SparseVolumeTexture from a dense source volume. Also did a little bit of renaming of some of the involved functions/interfaces. Removed unorm-remapping option when importing a .vdb file. The usefulness of this feature is debatable and it doesn't work properly on .vdb sequences.
...
#rb Sebastien.Hillaire
#rnx
#preflight 64469e6dc632d1e89b3dc367
#fyi Devon.Penney
[CL 25178321 by tim doerries in ue5-main branch]
2023-04-25 07:50:06 -04:00
tim doerries
19bb93a3fc
Implemented multi-phase initialization for SparseVolumeTextures.
...
#rb Devon.Penney
#rnx
#preflight 643848bcdf20983c4172fa18
[CL 25038431 by tim doerries in ue5-main branch]
2023-04-14 07:47:32 -04:00
tim doerries
bb1da72ead
Implemented support for importing .vdb grids with half float formats.
...
#rb Patrick.Kelly
#rnx
#preflight 6436b7e900398d6f88504765
[CL 25025416 by tim doerries in ue5-main branch]
2023-04-13 10:50:12 -04:00
tim doerries
2eb735892e
Added try/catch around openvdb file parsing to catch exceptions thrown on bad files. Moved grid typedefs for openvdb types into shared header.
...
#rb none
#rnx
#preflight 6436a9912909bc56c82e657a
[CL 25006193 by tim doerries in ue5-main branch]
2023-04-12 09:45:05 -04:00
tim doerries
19fe3af446
Moved SparseVolumeTexture initialization into a function, simplifying (and hiding implementation details of) SVT creation in factories etc.
...
#rb Sebastien.Hillaire
#rnx
#preflight 641db9ffc42c7fd1cbba3436
[CL 24781341 by tim doerries in ue5-main branch]
2023-03-24 12:42:28 -04:00
tim doerries
5bfb8ca971
Fix for imported SVT assets having a name different from the package name, which causes them to not show up in the content browser.
...
#rnx
#preflight trivial
[CL 24752141 by tim doerries in ue5-main branch]
2023-03-22 14:58:34 -04:00
tim doerries
ef19fb9547
Fixes for two small bugs when trying to import certain .vdb files. One is an off-by-one bug, the other one is an edge case for when a SVT frame is empty except for the null tile.
...
#rnx
#preflight trivial
[CL 24740075 by tim doerries in ue5-main branch]
2023-03-21 17:22:58 -04:00
tim doerries
5402186994
Fixed VDB importer missing tile data on import. Implemented SVT tile deduplication by hashing and redirecting page table entries to identical tiles, potentially saving a good chunk of memory on suitable assets.
...
#rb Sebastien.Hillaire
#rnx
#preflight 640b37dc3c2db40029943cd3
[CL 24592752 by tim doerries in ue5-main branch]
2023-03-10 11:43:31 -05:00
tim doerries
ed6155ff68
All SparseVolumeTexture mips are now stored in the same resource/proxy, making binding and sampling different mip levels much easier. This version is still missing streaming at mip level granularity and has a minimum mip level size of 16x16x16 (no mip tails). Since the added logic started bloating SparseVolumeTexture.h/.cpp, FSparseVolumeRawSource (now renamed to FSparseVolumeTextureData), FSparseVolumeTextureRuntime and FSparseVolumeTextureProxy were moved into their own files. FSparseVolumeTextureData stores a potentially mipped (after cooking) 3D page table indexing into linear arrays of physical tile data. At runtime the physical tiles are rearranged/packed into 3D physical tile textures in FSparseVolumeTextureRuntime and the page table payload adjusted from indices to 3D tile coordinates. The SVTSample node now also takes an optional MipLevel parameter which enables sampling across mips (lerp).
...
#rb Sebastien.Hillaire
#rnx
#preflight 63fdec5aae54ee4ce99490f9
[CL 24471451 by tim doerries in ue5-main branch]
2023-03-01 16:56:41 -05:00
tim doerries
cd143277a9
Fixed OpenVDB import being limited to a maximum source asset file size of 2GB.
...
#rb Sebastien.Hillaire
#rnx
#preflight 63f74973b91d7b0a122429cd
[CL 24386923 by tim doerries in ue5-main branch]
2023-02-23 14:53:04 -05:00
tim doerries
5f1e71bd18
Implemented first version of mip maps for SparseVolumeTextures. Each mip is currently an individual SVT structure/proxy. Mip generation is currently done on import; this will be moved to cooking in a later CL. Much of the OpenVDB->SVT conversion logic has been moved into SparseVolumeTexture.cpp, decoupling it from the vdb library. OpenVDBUtility.cpp now only implements an interface for querying data from the source file. The SVT preview received a few fixes for issues that became apparent when trying to visualize low res mips (Can't early out if the packed page table result is 0 when no null tile exists; ray stepping needs to be done after volume sampling in order to not miss voxels; sampling needs to be guarded by checking against the virtual volume resolution).
...
#rb Sebastien.Hillaire
#rnx
#preflight 63f5d9b2ac06ce789f025459
[CL 24360541 by tim doerries in ue5-main branch]
2023-02-22 09:39:24 -05:00
tim doerries
ba186ef2a0
Converted SparseVolumeTextures from AssetTypeActions to AssetDefinition.
...
#rnx
#preflight 63f5fdfc977ceed9152ec256
[FYI] Sebastien.Hillaire
[CL 24359383 by tim doerries in ue5-main branch]
2023-02-22 07:58:57 -05:00
tim doerries
ed3cc7ae76
Import of .vdb sequences for SparseVolumeTextures now uses multithreading to parallelize per-frame processing. Fixed background/fallback values always being zero, even if the value specified in the source .vdb file is non-zero.
...
#rb Sebastien.Hillaire
#rnx
#preflight 63ed0cf5205b5ccbc71dc82e
[CL 24258199 by tim doerries in ue5-main branch]
2023-02-16 10:42:36 -05:00
tim doerries
7f260f4daa
Implemented first version of streaming for SparseVolumeTextures. Fixed PostEditChange() being called twice on newly imported SVT. Small refactors on UStreamableSVT, UStaticSVT and UAnimatedSVT.
...
#rb Sebastien.Hillaire
#rnx
#preflight 63dd2dcf4d25c400b7b7cfaa
[CL 24047693 by tim doerries in ue5-main branch]
2023-02-07 08:59:18 -05:00
tim doerries
2bd647ec10
Switched SparseVolumeTextures to new DDC API. Also introduced UStreamableSparseVolumeTexture as new base class for static and animated SVT classes. This is in preparation for streaming and also to clean up and deduplicate some code that could be shared. Fixed missing call to ExpandVolumeBounds() when importing an animated SVT.
...
#rb Sebastien.Hillaire
#rnx
#preflight 63d9528a65738ba951d804a8
[CL 23945941 by tim doerries in ue5-main branch]
2023-02-01 06:20:15 -05:00
tim doerries
1ca91d6ea2
SVT uassets now no longer store the original vdb file. Conversion now happens during import, giving the user a progress bar instead of just blocking the editor on PostLoad() when the derived data was generated. Also renamed "PackedData" to "Attributes".
...
#rnx
#preflight 63d7f61031334253e5ff28c4
[CL 23923279 by tim doerries in ue5-main branch]
2023-01-31 01:11:48 -05:00
