izzy/UnrealEngineUWP
0
0
Fork 0
mirror of https://github.com/izzy2lost/UnrealEngineUWP.git synced 2026-03-26 18:15:20 -07:00
Code Issues Packages Projects Releases Wiki Activity
Files
5.3
UnrealEngineUWP/Engine/Plugins/Runtime/MeshModelingToolset/Source/ModelingComponents/Private/Scene/SceneCapturePhotoSet.cpp

960 lines
32 KiB
C++
Raw Permalink Normal View History

ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
// Copyright Epic Games, Inc. All Rights Reserved.
#include "Scene/SceneCapturePhotoSet.h"
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
#include "EngineUtils.h"
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
#include "Misc/ScopedSlowTask.h"
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
using namespace UE::Geometry;
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
#define LOCTEXT_NAMESPACE "SceneCapture"
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FSceneCapturePhotoSet::FSceneCapturePhotoSet()
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
// These defaults are chosen so an unfamiliar user doesn't need to call the configuration functions to get a result
ForEachCaptureType([this](ERenderCaptureType CaptureType)
{
bool bIsDisabled = (
CaptureType == ERenderCaptureType::DeviceDepth ||
CaptureType == ERenderCaptureType::Roughness ||
CaptureType == ERenderCaptureType::Specular ||
CaptureType == ERenderCaptureType::Metallic);
PhotoSetStatus[CaptureType] = bIsDisabled ? ECaptureTypeStatus::Disabled : ECaptureTypeStatus::Pending;
RenderCaptureConfig[CaptureType] = GetDefaultRenderCaptureConfig(CaptureType);
});
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
void FSceneCapturePhotoSet::SetCaptureSceneActors(UWorld* World, const TArray<AActor*>& Actors)
{
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
if (this->TargetWorld != World || this->VisibleActors != Actors)
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
ForEachCaptureType([this](ERenderCaptureType CaptureType)
{
if (PhotoSetStatus[CaptureType] != ECaptureTypeStatus::Disabled)
{
PhotoSetStatus[CaptureType] = ECaptureTypeStatus::Pending;
}
});
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
// Empty the photo sets because they rendered different actors
EmptyAllPhotoSets();
// Empty the spatial photo parameters because these are computed from the bounding box of the actors
PhotoSetParams.Empty();
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
this->TargetWorld = World;
this->VisibleActors = Actors;
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
TArray<AActor*> FSceneCapturePhotoSet::GetCaptureSceneActors()
{
return VisibleActors;
}
UWorld* FSceneCapturePhotoSet::GetCaptureTargetWorld()
{
return TargetWorld;
}
void FSceneCapturePhotoSet::SetSpatialPhotoParams(const TArray<FSpatialPhotoParams>& SpatialParams)
{
// TODO Discard/reset on a per array element level rather than discarding everything when any viewpoint changed
if (PhotoSetParams != SpatialParams)
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
ForEachCaptureType([this](ERenderCaptureType CaptureType)
{
if (PhotoSetStatus[CaptureType] != ECaptureTypeStatus::Disabled)
{
PhotoSetStatus[CaptureType] = ECaptureTypeStatus::Pending;
}
});
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
EmptyAllPhotoSets();
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
PhotoSetParams = SpatialParams;
}
}
const TArray<FSpatialPhotoParams>& FSceneCapturePhotoSet::GetSpatialPhotoParams() const
{
return PhotoSetParams;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
void FSceneCapturePhotoSet::SetCaptureConfig(ERenderCaptureType CaptureType, const FRenderCaptureConfig& NewConfig)
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (RenderCaptureConfig[CaptureType] != NewConfig)
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[CaptureType] != ECaptureTypeStatus::Disabled)
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
PhotoSetStatus[CaptureType] = ECaptureTypeStatus::Pending;
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
EmptyPhotoSet(CaptureType);
RenderCaptureConfig[CaptureType] = NewConfig;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
}
}
FRenderCaptureConfig FSceneCapturePhotoSet::GetCaptureConfig(ERenderCaptureType CaptureType) const
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
return RenderCaptureConfig[CaptureType];
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
void FSceneCapturePhotoSet::DisableAllCaptureTypes()
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
ForEachCaptureType([this](ERenderCaptureType CaptureType)
{
SetCaptureTypeEnabled(CaptureType, false);
});
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
void FSceneCapturePhotoSet::SetCaptureTypeEnabled(ERenderCaptureType CaptureType, bool bEnable)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (bEnable)
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[CaptureType] == ECaptureTypeStatus::Disabled)
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
PhotoSetStatus[CaptureType] = ECaptureTypeStatus::Pending;
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
}
else
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
PhotoSetStatus[CaptureType] = ECaptureTypeStatus::Disabled;
EmptyPhotoSet(CaptureType);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
FSceneCapturePhotoSet::ECaptureTypeStatus FSceneCapturePhotoSet::GetCaptureTypeStatus(ERenderCaptureType CaptureType) const
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
return PhotoSetStatus[CaptureType];
}
FSceneCapturePhotoSet::FStatus FSceneCapturePhotoSet::GetSceneCaptureStatus() const
{
return PhotoSetStatus;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
void FSceneCapturePhotoSet::Compute()
{
Fix ensure in BakeRC tool which occured when disabling a capture channel after cancelling a previous scene capture #rb lonnie.li #jira UE-191193 [CL 26609454 by matija kecman in 5.3 branch]
2023-07-26 11:49:00 -04:00
bWasCancelled = false;
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
int NumPending = 0;
ForEachCaptureType([this, &NumPending](ERenderCaptureType CaptureType)
{
NumPending += static_cast<int>(PhotoSetStatus[CaptureType] == ECaptureTypeStatus::Pending);
});
if (NumPending == 0)
{
return;
}
TRACE_CPUPROFILER_EVENT_SCOPE(CapturePhotoSet);
FScopedSlowTask Progress(static_cast<float>(NumPending), LOCTEXT("CapturingScene", "Capturing Scene..."));
Progress.MakeDialog(bAllowCancel);
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
check(this->TargetWorld != nullptr);
// Unregister all components to remove unwanted proxies from the scene. This was previously the only way to "hide" nanite meshes, now optional.
TSet<AActor*> VisibleActorsSet(VisibleActors);
TArray<AActor*> ActorsToRegister;
if (bEnforceVisibilityViaUnregister)
{
for (TActorIterator<AActor> Actor(TargetWorld); Actor; ++Actor)
{
if (!VisibleActorsSet.Contains(*Actor))
{
Actor->UnregisterAllComponents();
ActorsToRegister.Add(*Actor);
}
}
}
ON_SCOPE_EXIT
{
// Workaround for Nanite scene proxies visibility
// Reregister all components we previously unregistered
for (AActor* Actor : ActorsToRegister)
{
Actor->RegisterAllComponents();
}
};
FWorldRenderCapture RenderCapture;
RenderCapture.SetWorld(TargetWorld);
RenderCapture.SetVisibleActors(VisibleActors);
if (bWriteDebugImages)
{
RenderCapture.SetEnableWriteDebugImage(true, 0, DebugImagesFolderName);
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
auto CapturePhoto3f = [this, &RenderCapture](ERenderCaptureType CaptureType, const FSpatialPhotoParams& Params)
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
FSpatialPhoto3f NewPhoto;
NewPhoto.Frame = Params.Frame;
NewPhoto.NearPlaneDist = Params.NearPlaneDist;
NewPhoto.HorzFOVDegrees = Params.HorzFOVDegrees;
NewPhoto.Dimensions = Params.Dimensions;
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
// TODO Do something with the success boolean returned by RenderCapture.CaptureFromPosition
FImageAdapter Image(&NewPhoto.Image);
FRenderCaptureConfig Config = GetCaptureConfig(CaptureType);
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
RenderCapture.SetDimensions(Params.Dimensions);
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
RenderCapture.CaptureFromPosition(CaptureType, NewPhoto.Frame, NewPhoto.HorzFOVDegrees, NewPhoto.NearPlaneDist, Image, Config);
GetPhotoSet3f(CaptureType).Add(MoveTemp(NewPhoto));
};
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
auto CapturePhoto1f = [this, &RenderCapture](ERenderCaptureType CaptureType, const FSpatialPhotoParams& Params)
{
FSpatialPhoto1f NewPhoto;
NewPhoto.Frame = Params.Frame;
NewPhoto.NearPlaneDist = Params.NearPlaneDist;
NewPhoto.HorzFOVDegrees = Params.HorzFOVDegrees;
NewPhoto.Dimensions = Params.Dimensions;
// TODO Do something with the success boolean returned by RenderCapture.CaptureFromPosition
FImageAdapter Image(&NewPhoto.Image);
FRenderCaptureConfig Config = GetCaptureConfig(CaptureType);
RenderCapture.SetDimensions(Params.Dimensions);
RenderCapture.CaptureFromPosition(CaptureType, NewPhoto.Frame, NewPhoto.HorzFOVDegrees, NewPhoto.NearPlaneDist, Image, Config);
GetPhotoSet1f(CaptureType).Add(MoveTemp(NewPhoto));
if (CaptureType == ERenderCaptureType::DeviceDepth)
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
PhotoViewMatricies.Add(RenderCapture.GetLastCaptureViewMatrices());
}
};
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
// Iterate by channel computing all the photos rather than by photo/viewpoint computing all the channels.
// Note: This ordering means that some captures may have computed even if the computation is cancelled.
auto CapturePhotoSet = [this, &Progress](
TFunctionRef<void(ERenderCaptureType, const FSpatialPhotoParams&)> CapturePhoto,
ERenderCaptureType CaptureType,
const FText& TaskMessage)
{
if (PhotoSetStatus[CaptureType] == ECaptureTypeStatus::Pending)
{
Progress.EnterProgressFrame(1.f);
FScopedSlowTask PhotoSetProgress(PhotoSetParams.Num(), TaskMessage);
EmptyPhotoSet(CaptureType);
for (const FSpatialPhotoParams& Params : PhotoSetParams)
{
if (Progress.ShouldCancel())
{
UE_LOG(LogGeometry, Display, TEXT("FSceneCapturePhotoSet: The pending '%s' step was cancelled"), *TaskMessage.ToString());
bWasCancelled = true;
return;
}
PhotoSetProgress.EnterProgressFrame(1.f);
CapturePhoto(CaptureType, Params);
Progress.TickProgress();
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
PhotoSetStatus[CaptureType] = ECaptureTypeStatus::Computed;
}
};
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
Fix horde issue 337708 related to text conflicts in LOCTEXT #rb trivial #horde 337708 [CL 26601084 by matija kecman in 5.3 branch]
2023-07-26 04:48:13 -04:00
CapturePhotoSet(CapturePhoto1f, ERenderCaptureType::DeviceDepth, LOCTEXT("CapturingScene_DeviceDepth", "Capturing Device Depth"));
CapturePhotoSet(CapturePhoto3f, ERenderCaptureType::BaseColor, LOCTEXT("CapturingScene_BaseColor", "Capturing Base Color"));
CapturePhotoSet(CapturePhoto3f, ERenderCaptureType::WorldNormal, LOCTEXT("CapturingScene_WorldNormal", "Capturing World Normal"));
CapturePhotoSet(CapturePhoto3f, ERenderCaptureType::CombinedMRS, LOCTEXT("CapturingScene_CombinedMRS", "Capturing Packed MRS"));
CapturePhotoSet(CapturePhoto1f, ERenderCaptureType::Metallic, LOCTEXT("CapturingScene_Metallic", "Capturing Metallic"));
CapturePhotoSet(CapturePhoto1f, ERenderCaptureType::Roughness, LOCTEXT("CapturingScene_Roughness", "Capturing Roughness"));
CapturePhotoSet(CapturePhoto1f, ERenderCaptureType::Specular, LOCTEXT("CapturingScene_Specular", "Capturing Specular"));
CapturePhotoSet(CapturePhoto3f, ERenderCaptureType::Emissive, LOCTEXT("CapturingScene_Emissive", "Capturing Emissive"));
CapturePhotoSet(CapturePhoto1f, ERenderCaptureType::Opacity, LOCTEXT("CapturingScene_Opacity", "Capturing Opacity"));
CapturePhotoSet(CapturePhoto3f, ERenderCaptureType::SubsurfaceColor, LOCTEXT("CapturingScene_SubsurfaceColor", "Capturing Subsurface Color"));
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
void FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(
FImageDimensions PhotoDimensions,
double HorizontalFOVDegrees,
double NearPlaneDist,
bool bFaces,
bool bUpperCorners,
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
bool bLowerCorners,
bool bUpperEdges,
bool bSideEdges)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
TArray<FVector3d> Directions;
if (bFaces)
{
Directions.Add(FVector3d::UnitX());
Directions.Add(-FVector3d::UnitX());
Directions.Add(FVector3d::UnitY());
Directions.Add(-FVector3d::UnitY());
Directions.Add(FVector3d::UnitZ());
Directions.Add(-FVector3d::UnitZ());
}
if (bUpperCorners)
{
Directions.Add(Normalized(FVector3d(1, 1, -1)));
Directions.Add(Normalized(FVector3d(-1, 1, -1)));
Directions.Add(Normalized(FVector3d(1, -1, -1)));
Directions.Add(Normalized(FVector3d(-1, -1, -1)));
}
if (bLowerCorners)
{
Directions.Add(Normalized(FVector3d(1, 1, 1)));
Directions.Add(Normalized(FVector3d(-1, 1, 1)));
Directions.Add(Normalized(FVector3d(1, -1, 1)));
Directions.Add(Normalized(FVector3d(-1, -1, 1)));
}
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
if (bUpperEdges)
{
Directions.Add(Normalized(FVector3d(-1, 0, -1)));
Directions.Add(Normalized(FVector3d(1, 0, -1)));
Directions.Add(Normalized(FVector3d(0, -1, -1)));
Directions.Add(Normalized(FVector3d(0, 1, -1)));
}
if (bSideEdges)
{
Directions.Add(Normalized(FVector3d(1, 1, 0)));
Directions.Add(Normalized(FVector3d(-1, 1, 0)));
Directions.Add(Normalized(FVector3d(1, -1, 0)));
Directions.Add(Normalized(FVector3d(-1, -1, 0)));
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
PRAGMA_DISABLE_DEPRECATION_WARNINGS
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
AddExteriorCaptures(PhotoDimensions, HorizontalFOVDegrees, NearPlaneDist, Directions);
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
PRAGMA_ENABLE_DEPRECATION_WARNINGS
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
void FSceneCapturePhotoSet::AddExteriorCaptures(
FImageDimensions PhotoDimensions,
double HorizontalFOVDegrees,
double NearPlaneDist,
const TArray<FVector3d>& Directions)
{
check(this->TargetWorld != nullptr);
Render Capture Baking: Fix a crash which occurred because the SceneCapture state was not properly updated when its computation was cancelled #jira UE-175047 #preflight 63d94591f626715201e45242 [CL 23933594 by matija kecman in ue5-main branch]
2023-01-31 14:12:46 -05:00
bWasCancelled = false;
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
FScopedSlowTask Progress(Directions.Num(), LOCTEXT("ComputingViewpoints", "Computing Viewpoints..."));
Progress.MakeDialog(bAllowCancel);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
// Unregister all components to remove unwanted proxies from the scene. This was previously the only way to "hide" nanite meshes, now optional.
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
TSet<AActor*> VisibleActorsSet(VisibleActors);
TArray<AActor*> ActorsToRegister;
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
if (bEnforceVisibilityViaUnregister)
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
{
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
for (TActorIterator<AActor> Actor(TargetWorld); Actor; ++Actor)
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
{
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
if (!VisibleActorsSet.Contains(*Actor))
{
Actor->UnregisterAllComponents();
ActorsToRegister.Add(*Actor);
}
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
}
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
ON_SCOPE_EXIT
{
// Workaround for Nanite scene proxies visibility
// Reregister all components we previously unregistered
for (AActor* Actor : ActorsToRegister)
{
Actor->RegisterAllComponents();
}
};
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
FWorldRenderCapture RenderCapture;
RenderCapture.SetWorld(TargetWorld);
RenderCapture.SetVisibleActors(VisibleActors);
RenderCapture.SetDimensions(PhotoDimensions);
GeometryProcessing: - extend FImageAdapter to support reading ModelingComponents:: - Add AssetUtils::ForceVirtualTexturePrefetch(), ::SaveDebugImage() variants to Texture2DUtil - Add support for debug image writing to WorldRenderCapture and SceneCapturePhotoSet - FWorldRenderCapture now computes more accurate bounds (visible Components instead of Actors), and does an explicit VirtualTexture prefetch before doing render capture - FStaticMeshAssetOptions now exposes Nanite settings to allow UStaticMeshes to be created with Nanite already enabled (currently NaniteSettings.PositionPrecision defaults to max) ApproximateActors: - IGeometryProcessing_ApproximateActors::FOptions now has settings for enabling Nanite on generated meshes - ApproximateActorsImpl applies these settings, also enables VT on generated textures if necessary - FMeshApproximationSettings/FMeshApproximationTool updated w/ new Nanite settings #rb none #rnx #jira none [FYI] sebastien.lussier #preflight 60b08323f51cd90001775b98 #ROBOMERGE-OWNER: ryan.schmidt #ROBOMERGE-AUTHOR: ryan.schmidt #ROBOMERGE-SOURCE: CL 16499613 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v823-16466674) #ROBOMERGE-CONFLICT from-shelf [CL 16499647 by ryan schmidt in ue5-main branch]
2021-05-28 02:46:59 -04:00
if (bWriteDebugImages)
{
RenderCapture.SetEnableWriteDebugImage(true, 0, DebugImagesFolderName);
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
// this tells us origin and radius - could be view-dependent...
FSphere RenderSphere = RenderCapture.ComputeContainingRenderSphere(HorizontalFOVDegrees);
int32 NumDirections = Directions.Num();
for (int32 di = 0; di < NumDirections; ++di)
{
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
Progress.EnterProgressFrame(1.f);
if (Progress.ShouldCancel())
{
bWasCancelled = true;
return;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
FVector3d ViewDirection = Directions[di];
ViewDirection.Normalize();
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
FSpatialPhotoParams Params;
Params.NearPlaneDist = NearPlaneDist;
Params.HorzFOVDegrees = HorizontalFOVDegrees;
Params.Dimensions = PhotoDimensions;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
// TODO Align the frame with the renderer coordinate system then remove the axis swapping in WorldRenderCapture.cpp
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
Params.Frame.AlignAxis(0, ViewDirection);
Params.Frame.ConstrainedAlignAxis(2, FVector3d::UnitZ(), Params.Frame.X());
Params.Frame.Origin = RenderSphere.Center;
Params.Frame.Origin -= RenderSphere.W * Params.Frame.X();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
auto CaptureImageTypeFunc_3f = [this, &Progress, &RenderCapture, &Params, &NumDirections]
(ERenderCaptureType CaptureType, FSpatialPhotoSet3f& PhotoSet)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
// Test NumDirections to avoid recomputing photo sets. Search :SceneCaptureWithExistingCaptures
if (PhotoSet.Num() < NumDirections)
{
FSpatialPhoto3f NewPhoto;
NewPhoto.Frame = Params.Frame;
NewPhoto.NearPlaneDist = Params.NearPlaneDist;
NewPhoto.HorzFOVDegrees = Params.HorzFOVDegrees;
NewPhoto.Dimensions = Params.Dimensions;
// TODO Do something with the success boolean returned by RenderCapture.CaptureFromPosition
FImageAdapter Image(&NewPhoto.Image);
FRenderCaptureConfig Config = GetCaptureConfig(CaptureType);
RenderCapture.CaptureFromPosition(CaptureType, NewPhoto.Frame, NewPhoto.HorzFOVDegrees, NewPhoto.NearPlaneDist, Image, Config);
PhotoSet.Add(MoveTemp(NewPhoto));
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
Progress.TickProgress();
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
};
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
auto CaptureImageTypeFunc_1f = [this, &Progress, &RenderCapture, &Params, &NumDirections]
(ERenderCaptureType CaptureType, FSpatialPhotoSet1f& PhotoSet)
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
{
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
// Test NumDirections to avoid recomputing photo sets. Search :SceneCaptureWithExistingCaptures
if (PhotoSet.Num() < NumDirections)
{
FSpatialPhoto1f NewPhoto;
NewPhoto.Frame = Params.Frame;
NewPhoto.NearPlaneDist = Params.NearPlaneDist;
NewPhoto.HorzFOVDegrees = Params.HorzFOVDegrees;
NewPhoto.Dimensions = Params.Dimensions;
// TODO Do something with the success boolean returned by RenderCapture.CaptureFromPosition
FImageAdapter Image(&NewPhoto.Image);
FRenderCaptureConfig Config = GetCaptureConfig(CaptureType);
RenderCapture.CaptureFromPosition(CaptureType, NewPhoto.Frame, NewPhoto.HorzFOVDegrees, NewPhoto.NearPlaneDist, Image, Config);
PhotoSet.Add(MoveTemp(NewPhoto));
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
Progress.TickProgress();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
};
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::DeviceDepth] == ECaptureTypeStatus::Pending)
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
{
CaptureImageTypeFunc_1f(ERenderCaptureType::DeviceDepth, DeviceDepthPhotoSet);
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
if (PhotoViewMatricies.Num() < NumDirections)
{
PhotoViewMatricies.Add(RenderCapture.GetLastCaptureViewMatrices());
}
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::BaseColor] == ECaptureTypeStatus::Pending)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_3f(ERenderCaptureType::BaseColor, BaseColorPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::Roughness] == ECaptureTypeStatus::Pending)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_1f(ERenderCaptureType::Roughness, RoughnessPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::Specular] == ECaptureTypeStatus::Pending)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_1f(ERenderCaptureType::Specular, SpecularPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::Metallic] == ECaptureTypeStatus::Pending)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_1f(ERenderCaptureType::Metallic, MetallicPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::CombinedMRS] == ECaptureTypeStatus::Pending)
ModelingComponents: support Packed Metallic/Roughness/Specular (MRS) capture in WorldRenderCapture using custom postprocess render pass. Also support in FSceneCapturePhotoSet. Add Linear Color variant to FTexture2DBuilder. #rb none #rnx #jira none #preflight 60ade5d11d02bb0001883281 #ROBOMERGE-SOURCE: CL 16465354 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v818-16446889) [CL 16465367 by ryan schmidt in ue5-main branch]
2021-05-26 03:02:11 -04:00
{
CaptureImageTypeFunc_3f(ERenderCaptureType::CombinedMRS, PackedMRSPhotoSet);
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::WorldNormal] == ECaptureTypeStatus::Pending)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_3f(ERenderCaptureType::WorldNormal, WorldNormalPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::Emissive] == ECaptureTypeStatus::Pending)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_3f(ERenderCaptureType::Emissive, EmissivePhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::Opacity] == ECaptureTypeStatus::Pending)
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
{
CaptureImageTypeFunc_1f(ERenderCaptureType::Opacity, OpacityPhotoSet);
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
if (PhotoSetStatus[ERenderCaptureType::SubsurfaceColor] == ECaptureTypeStatus::Pending)
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
{
CaptureImageTypeFunc_3f(ERenderCaptureType::SubsurfaceColor, SubsurfaceColorPhotoSet);
}
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
// AddExteriorCaptures can be called on an FSceneCapturePhotoSet which already has some capture types computed
// and in this case we should not modify the existing photo sets/cached photo set parameters.
// See :SceneCaptureWithExistingCaptures
if (PhotoSetParams.Num() < NumDirections)
{
PhotoSetParams.Add(Params);
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
} // end directions loop
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
void FSceneCapturePhotoSet::OptimizePhotoSets()
{
// todo:
// 1) crop photos to regions with actual pixels
// 2) pack into fewer photos (eg pack spec/rough/metallic)
// 3) RLE encoding or other compression
}
FSceneCapturePhotoSet::FSceneSample::FSceneSample()
{
HaveValues = FRenderCaptureTypeFlags::None();
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
BaseColor = FVector3f(0, 0, 0);
Roughness = 0.0f;
Specular = 0.0f;
Metallic = 0.0f;
Emissive = FVector3f(0, 0, 0);
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
Opacity = 0.0f;
SubsurfaceColor = FVector3f(0, 0, 0);
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
WorldNormal = FVector3f(0, 0, 1);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
DeviceDepth = 0.0f;
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
FVector3f FSceneCapturePhotoSet::FSceneSample::GetValue3f(ERenderCaptureType CaptureType) const
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
return BaseColor;
case ERenderCaptureType::WorldNormal:
return WorldNormal;
case ERenderCaptureType::Roughness:
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
return Roughness * FVector3f::One();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
case ERenderCaptureType::Metallic:
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
return Metallic * FVector3f::One();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
case ERenderCaptureType::Specular:
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
return Specular * FVector3f::One();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
case ERenderCaptureType::Emissive:
return Emissive;
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
case ERenderCaptureType::Opacity:
return Opacity * FVector3f::One();
case ERenderCaptureType::SubsurfaceColor:
return SubsurfaceColor;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
case ERenderCaptureType::DeviceDepth:
return DeviceDepth * FVector3f::One();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
default:
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
ensure(false);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
return FVector3f::Zero();
}
FVector4f FSceneCapturePhotoSet::FSceneSample::GetValue4f(ERenderCaptureType CaptureType) const
{
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
return FVector4f(BaseColor.X, BaseColor.Y, BaseColor.Z, 1.0f);
case ERenderCaptureType::WorldNormal:
return FVector4f(WorldNormal.X, WorldNormal.Y, WorldNormal.Z, 1.0f);
case ERenderCaptureType::Roughness:
return FVector4f(Roughness, Roughness, Roughness, 1.0f);
case ERenderCaptureType::Metallic:
return FVector4f(Metallic, Metallic, Metallic, 1.0f);
case ERenderCaptureType::Specular:
return FVector4f(Specular, Specular, Specular, 1.0f);
ModelingComponents: support Packed Metallic/Roughness/Specular (MRS) capture in WorldRenderCapture using custom postprocess render pass. Also support in FSceneCapturePhotoSet. Add Linear Color variant to FTexture2DBuilder. #rb none #rnx #jira none #preflight 60ade5d11d02bb0001883281 #ROBOMERGE-SOURCE: CL 16465354 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v818-16446889) [CL 16465367 by ryan schmidt in ue5-main branch]
2021-05-26 03:02:11 -04:00
case ERenderCaptureType::CombinedMRS:
return FVector4f(Metallic, Roughness, Specular, 1.0f);
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
case ERenderCaptureType::Emissive:
return FVector4f(Emissive.X, Emissive.Y, Emissive.Z, 1.0f);
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
case ERenderCaptureType::Opacity:
return FVector4f(Opacity, Opacity, Opacity, 1.0f);
case ERenderCaptureType::SubsurfaceColor:
return FVector4f(SubsurfaceColor.X, SubsurfaceColor.Y, SubsurfaceColor.Z, 1.0f);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
case ERenderCaptureType::DeviceDepth:
return FVector4f(DeviceDepth, DeviceDepth, DeviceDepth, 1.0f);
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
default:
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
ensure(false);
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
return FVector4f::Zero();
}
Render Capture Baking: Move visibility raycast from evaluators to detail sampler Benchmark configuration: - Test scene with 4 meshes: a mossy stone nanite mesh, a beach ball, a ball with an emissive blue cross and a hut mesh - Target mesh: 11.8k triangles mesh, created by voxel wrapping the meshes in the test scene elements - 4096x4096 output texture with 16x multisampling baking BaseColor, Emissive, PackedMRS and Normal maps - 4096x4096 photo capture resolution Unreal Insights reports the following times for the Bake call: - Before this change: 2 mins 45 seconds - After this change: 55.9 seconds Aside: (Unaffected by this change) In both cases the photo capture, which is done on the game thread, took about 1 min #jira none #rb #rnx #preflight 627e64e01748fbc85bb24a42 [CL 20221383 by matija kecman in ue5-main branch]
2022-05-16 05:13:25 -04:00
bool FSceneCapturePhotoSet::ComputeSampleLocation(
const FVector3d& Position,
const FVector3d& Normal,
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
const float ValidSampleDepthThreshold,
Render Capture Baking: Move visibility raycast from evaluators to detail sampler Benchmark configuration: - Test scene with 4 meshes: a mossy stone nanite mesh, a beach ball, a ball with an emissive blue cross and a hut mesh - Target mesh: 11.8k triangles mesh, created by voxel wrapping the meshes in the test scene elements - 4096x4096 output texture with 16x multisampling baking BaseColor, Emissive, PackedMRS and Normal maps - 4096x4096 photo capture resolution Unreal Insights reports the following times for the Bake call: - Before this change: 2 mins 45 seconds - After this change: 55.9 seconds Aside: (Unaffected by this change) In both cases the photo capture, which is done on the game thread, took about 1 min #jira none #rb #rnx #preflight 627e64e01748fbc85bb24a42 [CL 20221383 by matija kecman in ue5-main branch]
2022-05-16 05:13:25 -04:00
TFunctionRef<bool(const FVector3d&, const FVector3d&)> VisibilityFunction,
int& PhotoIndex,
FVector2d& PhotoCoords) const
{
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
double DotTolerance = -0.1; // dot should be negative for normal pointing towards photo
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
if (ValidSampleDepthThreshold > 0)
{
check(DeviceDepthPhotoSet.Num() == PhotoSetParams.Num());
check(PhotoViewMatricies.Num() == PhotoSetParams.Num());
}
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
PhotoIndex = IndexConstants::InvalidID;
PhotoCoords = FVector2d(0., 0.);
double MinDot = 1.0;
int32 NumPhotos = PhotoSetParams.Num();
Fix Editor Mac which broke in Horde Issue #180297: Compile errors in Module.ModelingComponents.cpp and SceneCapturePhotoSet.cpp #rnx #rb jimmy.andrews #jira none #preflight 62a7511aaf7ad033effcec61 #preflight 62a7511aaf7ad033effcec61 [CL 20627748 by matija kecman in ue5-main branch]
2022-06-13 11:19:07 -04:00
for (int32 Index = 0; Index < NumPhotos; ++Index)
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
{
Fix Editor Mac which broke in Horde Issue #180297: Compile errors in Module.ModelingComponents.cpp and SceneCapturePhotoSet.cpp #rnx #rb jimmy.andrews #jira none #preflight 62a7511aaf7ad033effcec61 #preflight 62a7511aaf7ad033effcec61 [CL 20627748 by matija kecman in ue5-main branch]
2022-06-13 11:19:07 -04:00
const FSpatialPhotoParams& Params = PhotoSetParams[Index];
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
check(Params.Dimensions.IsSquare());
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FFrame3d RenderFrame(Params.Frame.Origin, Params.Frame.Y(), Params.Frame.Z(), Params.Frame.X());
FVector3d ViewDirection = RenderFrame.Z();
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
double ViewDot = ViewDirection.Dot(Normal);
if (ViewDot > DotTolerance || ViewDot > MinDot)
{
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
// The sample is facing away from the photo, or we found a photo more aligned with this sample
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
continue;
}
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FFrame3d ViewPlane = RenderFrame;
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
ViewPlane.Origin += Params.NearPlaneDist * ViewDirection;
double ViewPlaneWidthWorld = Params.NearPlaneDist * FMathd::Tan(Params.HorzFOVDegrees * 0.5 * FMathd::DegToRad);
double ViewPlaneHeightWorld = ViewPlaneWidthWorld;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
// Shoot a ray from the camera position toward the sample position and find the hit point on photo plane
constexpr int NormalAxisIndex = 2;
FVector3d RayOrigin = RenderFrame.Origin;
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
FVector3d RayDir = Normalized(Position - RayOrigin);
FVector3d HitPoint;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
bool bHit = ViewPlane.RayPlaneIntersection(RayOrigin, RayDir, NormalAxisIndex, HitPoint);
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
if (bHit)
{
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FVector2d DeviceXY;
DeviceXY.X = (HitPoint - ViewPlane.Origin).Dot(ViewPlane.X()) / ViewPlaneWidthWorld;
DeviceXY.Y = (HitPoint - ViewPlane.Origin).Dot(ViewPlane.Y()) / ViewPlaneHeightWorld;
if (FMathd::Abs(DeviceXY.X) < 1 && FMathd::Abs(DeviceXY.Y) < 1)
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
{
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
// Shoot a ray from the sample position toward the camera position checking occlusion
bool bVisible = VisibilityFunction(Position, RayOrigin);
if (bVisible)
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
{
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FVector2d UVCoords = FRenderCaptureCoordinateConverter2D::DeviceToUV(DeviceXY);
if (ValidSampleDepthThreshold > 0)
{
// Look up the device depth from the depth photo set, these values are from 0 (far plane) to
// 1 (near plane). We skip points which would unproject to the far plane, which is positioned at
// infinity, we also do not interpolate the depth values, doing so is not a good approximation
// of the underlying scene
float DeviceZ = DeviceDepthPhotoSet.Get(Index).Image.NearestSampleUV(UVCoords);
if (DeviceZ > 0)
{
// Compute the pixel position in world space to use it to compute a depth according to the render
FVector3d PixelPositionDevice{DeviceXY, DeviceZ};
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
FVector4d PixelPositionWorld = PhotoViewMatricies[Index].GetInvViewProjectionMatrix().TransformPosition(PixelPositionDevice);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
PixelPositionWorld /= PixelPositionWorld.W;
// Compare the depth of the sample with the depth of the pixel and consider the sample invalid
// if these do not match closely enough. This fixes artefacts which occur when sample ray just
// misses an obstruction and hits a pixel where the color was set by a slightly different ray,
// through the pixel center, which does hit the obstruction. This problem occurs because the
// depth capture was obtained by renderering the the source meshes but the visiblity function
// works on the target mesh
float PixelDepth = (RayOrigin - FVector3d(PixelPositionWorld)).Length();
float SampleDepth = (RayOrigin - Position).Length();
if (FMath::Abs(PixelDepth - SampleDepth) < ValidSampleDepthThreshold)
{
PhotoCoords.X = UVCoords.X * (double)Params.Dimensions.GetWidth();
PhotoCoords.Y = UVCoords.Y * (double)Params.Dimensions.GetHeight();
PhotoIndex = Index;
MinDot = ViewDot;
}
} // Test DeviceZ > 0
}
else
{
PhotoCoords.X = UVCoords.X * (double)Params.Dimensions.GetWidth();
PhotoCoords.Y = UVCoords.Y * (double)Params.Dimensions.GetHeight();
PhotoIndex = Index;
MinDot = ViewDot;
}
} // Test bVisible
} // Test UVCoords in (-1,1)x(-1,1)
} // Hit photo plane
} // Photo loop
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
return PhotoIndex != IndexConstants::InvalidID;
Render Capture Baking: Move visibility raycast from evaluators to detail sampler Benchmark configuration: - Test scene with 4 meshes: a mossy stone nanite mesh, a beach ball, a ball with an emissive blue cross and a hut mesh - Target mesh: 11.8k triangles mesh, created by voxel wrapping the meshes in the test scene elements - 4096x4096 output texture with 16x multisampling baking BaseColor, Emissive, PackedMRS and Normal maps - 4096x4096 photo capture resolution Unreal Insights reports the following times for the Bake call: - Before this change: 2 mins 45 seconds - After this change: 55.9 seconds Aside: (Unaffected by this change) In both cases the photo capture, which is done on the game thread, took about 1 min #jira none #rb #rnx #preflight 627e64e01748fbc85bb24a42 [CL 20221383 by matija kecman in ue5-main branch]
2022-05-16 05:13:25 -04:00
}
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
bool FSceneCapturePhotoSet::ComputeSample(
const FRenderCaptureTypeFlags& SampleChannels,
const FVector3d& Position,
const FVector3d& Normal,
TFunctionRef<bool(const FVector3d&, const FVector3d&)> VisibilityFunction,
FSceneSample& DefaultsInResultsOut) const
{
// This could be much more efficient if (eg) we knew that all the photo sets have
// the same captures, then the query only has to be done once and can be used to sample each specific photo
Render Capture Baking: Move visibility raycast from evaluators to detail sampler Benchmark configuration: - Test scene with 4 meshes: a mossy stone nanite mesh, a beach ball, a ball with an emissive blue cross and a hut mesh - Target mesh: 11.8k triangles mesh, created by voxel wrapping the meshes in the test scene elements - 4096x4096 output texture with 16x multisampling baking BaseColor, Emissive, PackedMRS and Normal maps - 4096x4096 photo capture resolution Unreal Insights reports the following times for the Bake call: - Before this change: 2 mins 45 seconds - After this change: 55.9 seconds Aside: (Unaffected by this change) In both cases the photo capture, which is done on the game thread, took about 1 min #jira none #rb #rnx #preflight 627e64e01748fbc85bb24a42 [CL 20221383 by matija kecman in ue5-main branch]
2022-05-16 05:13:25 -04:00
// This is implemented in the other ComputeSample overload
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
if (SampleChannels.bBaseColor)
{
DefaultsInResultsOut.BaseColor =
BaseColorPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.BaseColor);
DefaultsInResultsOut.HaveValues.bBaseColor = true;
}
if (SampleChannels.bRoughness)
{
DefaultsInResultsOut.Roughness =
RoughnessPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Roughness);
DefaultsInResultsOut.HaveValues.bRoughness = true;
}
if (SampleChannels.bSpecular)
{
DefaultsInResultsOut.Specular =
SpecularPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Specular);
DefaultsInResultsOut.HaveValues.bSpecular = true;
}
if (SampleChannels.bMetallic)
{
DefaultsInResultsOut.Metallic =
MetallicPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Metallic);
DefaultsInResultsOut.HaveValues.bMetallic = true;
}
ModelingComponents: support Packed Metallic/Roughness/Specular (MRS) capture in WorldRenderCapture using custom postprocess render pass. Also support in FSceneCapturePhotoSet. Add Linear Color variant to FTexture2DBuilder. #rb none #rnx #jira none #preflight 60ade5d11d02bb0001883281 #ROBOMERGE-SOURCE: CL 16465354 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v818-16446889) [CL 16465367 by ryan schmidt in ue5-main branch]
2021-05-26 03:02:11 -04:00
if (SampleChannels.bCombinedMRS)
{
FVector3f MRSValue(DefaultsInResultsOut.Metallic, DefaultsInResultsOut.Roughness, DefaultsInResultsOut.Specular);
MRSValue = PackedMRSPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, MRSValue);
DefaultsInResultsOut.Metallic = MRSValue.X;
DefaultsInResultsOut.Roughness = MRSValue.Y;
DefaultsInResultsOut.Specular = MRSValue.Z;
DefaultsInResultsOut.HaveValues.bMetallic = true;
DefaultsInResultsOut.HaveValues.bRoughness = true;
DefaultsInResultsOut.HaveValues.bSpecular = true;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
if (SampleChannels.bEmissive)
{
DefaultsInResultsOut.Emissive =
EmissivePhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Emissive);
DefaultsInResultsOut.HaveValues.bEmissive = true;
}
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
if (SampleChannels.bOpacity)
{
DefaultsInResultsOut.Opacity =
OpacityPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Opacity);
DefaultsInResultsOut.HaveValues.bOpacity = true;
}
if (SampleChannels.bSubsurfaceColor)
{
DefaultsInResultsOut.SubsurfaceColor =
SubsurfaceColorPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.SubsurfaceColor);
DefaultsInResultsOut.HaveValues.bSubsurfaceColor = true;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
if (SampleChannels.bWorldNormal)
{
DefaultsInResultsOut.WorldNormal =
WorldNormalPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.WorldNormal);
DefaultsInResultsOut.HaveValues.bWorldNormal = true;
}
return true;
GeometryProcessing: - extend FImageAdapter to support reading ModelingComponents:: - Add AssetUtils::ForceVirtualTexturePrefetch(), ::SaveDebugImage() variants to Texture2DUtil - Add support for debug image writing to WorldRenderCapture and SceneCapturePhotoSet - FWorldRenderCapture now computes more accurate bounds (visible Components instead of Actors), and does an explicit VirtualTexture prefetch before doing render capture - FStaticMeshAssetOptions now exposes Nanite settings to allow UStaticMeshes to be created with Nanite already enabled (currently NaniteSettings.PositionPrecision defaults to max) ApproximateActors: - IGeometryProcessing_ApproximateActors::FOptions now has settings for enabling Nanite on generated meshes - ApproximateActorsImpl applies these settings, also enables VT on generated textures if necessary - FMeshApproximationSettings/FMeshApproximationTool updated w/ new Nanite settings #rb none #rnx #jira none [FYI] sebastien.lussier #preflight 60b08323f51cd90001775b98 #ROBOMERGE-OWNER: ryan.schmidt #ROBOMERGE-AUTHOR: ryan.schmidt #ROBOMERGE-SOURCE: CL 16499613 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v823-16466674) #ROBOMERGE-CONFLICT from-shelf [CL 16499647 by ryan schmidt in ue5-main branch]
2021-05-28 02:46:59 -04:00
}
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
void FSceneCapturePhotoSet::SetEnableVisibilityByUnregisterMode(bool bEnable)
{
bEnforceVisibilityViaUnregister = bEnable;
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
void FSceneCapturePhotoSet::EmptyAllPhotoSets()
{
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
ForEachCaptureType([this](ERenderCaptureType CaptureType)
{
EmptyPhotoSet(CaptureType);
});
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
}
GeometryProcessing: - extend FImageAdapter to support reading ModelingComponents:: - Add AssetUtils::ForceVirtualTexturePrefetch(), ::SaveDebugImage() variants to Texture2DUtil - Add support for debug image writing to WorldRenderCapture and SceneCapturePhotoSet - FWorldRenderCapture now computes more accurate bounds (visible Components instead of Actors), and does an explicit VirtualTexture prefetch before doing render capture - FStaticMeshAssetOptions now exposes Nanite settings to allow UStaticMeshes to be created with Nanite already enabled (currently NaniteSettings.PositionPrecision defaults to max) ApproximateActors: - IGeometryProcessing_ApproximateActors::FOptions now has settings for enabling Nanite on generated meshes - ApproximateActorsImpl applies these settings, also enables VT on generated textures if necessary - FMeshApproximationSettings/FMeshApproximationTool updated w/ new Nanite settings #rb none #rnx #jira none [FYI] sebastien.lussier #preflight 60b08323f51cd90001775b98 #ROBOMERGE-OWNER: ryan.schmidt #ROBOMERGE-AUTHOR: ryan.schmidt #ROBOMERGE-SOURCE: CL 16499613 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v823-16466674) #ROBOMERGE-CONFLICT from-shelf [CL 16499647 by ryan schmidt in ue5-main branch]
2021-05-28 02:46:59 -04:00
void FSceneCapturePhotoSet::SetEnableWriteDebugImages(bool bEnable, FString FolderName)
{
bWriteDebugImages = bEnable;
if (FolderName.Len() > 0)
{
DebugImagesFolderName = FolderName;
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
}
Fix crashes in BakeRC tool, improved the scene capture progress bar, improved and simplified the scene capture cancellation behavior * Fixed a crash that occurred in the baking step when cancelling a scene capture which was triggered by changing an option which invalidated all photo sets (i.e., render capture resolution or capture anti aliasing). This was occurring because the baking step proceeded after the scene capture cancellation and the needed photosets were empty. * Fixed a crash which can happen if you have no channel enabled, change cleanup tolerance from 0 to 1 and then enable the BaseColor channel. This was fixed by changing the behavior of the Cleanup Threshold property, if its positive the DeviceDepth capture is enabled and its disabled otherwise, previously we also required that another channel was enabled which led to the crash in the baking step. The new semantics are easier to understand in the code and and is probably an improvement to the UX as well since the Cleanup Threshold property behavior does not depend on other properties. * Improved the scene capture cancellation: Users can now cancel the scene capture when the tool is starting up which is better UX since if they picked slow settings they don't intend to reuse, the capture can be cancelled, also, the code is simpler. Changed the behavior of the BakeRC tool properties so that after cancellation the completed captures with the desired settings are enabled. The old behavior where cancellation was documented to restore old results was broken and cannot be made to work unless we store the old results in a separate scene capture but for high capture resolutions that would require a lot of memory. When the SceneCapture is cancelled the channels which were pending and didn't complete are logged in the Output Log in case the user forgets which ones were enabled. * Simplified the code in various places: Added `operator[](ERenderCaptureType)` to structs with per-capture data, and use it to simplify many places where we previously checked channels individually. Added conversion functions from tool properties/geometry script parameters to the render capture parameters. Removed old confusing logic that determined if a capture had been updated by tracking the number of photos in each photoset with a new system based on a new per-capture pending state. Deprecated a bunch of functions which are no longer useful or which used the wrong types (e.g., `FRenderCaptureUpdate` was added because I hadn't noticed `FRenderCaptureTypeFlags`). Changed the defaults in `FSceneCapturePhotoSet` so that the DeviceDepth capture is off, its an advanced feature and the defaults are intended to make the class friendly to new developers #rb lonnie.li [CL 26568190 by matija kecman in 5.3 branch]
2023-07-25 05:08:58 -04:00
FSpatialPhotoSet1f& FSceneCapturePhotoSet::GetPhotoSet1f(ERenderCaptureType CaptureType)
{
switch (CaptureType)
{
case ERenderCaptureType::Roughness:
return RoughnessPhotoSet;
case ERenderCaptureType::Metallic:
return MetallicPhotoSet;
case ERenderCaptureType::Specular:
return SpecularPhotoSet;
case ERenderCaptureType::Opacity:
return OpacityPhotoSet;
case ERenderCaptureType::DeviceDepth:
return DeviceDepthPhotoSet;
default:
ensure(false);
}
return RoughnessPhotoSet;
}
FSpatialPhotoSet3f& FSceneCapturePhotoSet::GetPhotoSet3f(ERenderCaptureType CaptureType)
{
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
return BaseColorPhotoSet;
case ERenderCaptureType::CombinedMRS:
return PackedMRSPhotoSet;
case ERenderCaptureType::WorldNormal:
return WorldNormalPhotoSet;
case ERenderCaptureType::Emissive:
return EmissivePhotoSet;
case ERenderCaptureType::SubsurfaceColor:
return SubsurfaceColorPhotoSet;
default:
ensure(false);
}
return BaseColorPhotoSet;
}
void FSceneCapturePhotoSet::EmptyPhotoSet(ERenderCaptureType CaptureType)
{
// Set the functions to empty the photo sets
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
BaseColorPhotoSet.Empty();
break;
case ERenderCaptureType::WorldNormal:
WorldNormalPhotoSet.Empty();
break;
case ERenderCaptureType::Roughness:
RoughnessPhotoSet.Empty();
break;
case ERenderCaptureType::Metallic:
MetallicPhotoSet.Empty();
break;
case ERenderCaptureType::Specular:
SpecularPhotoSet.Empty();
break;
case ERenderCaptureType::Emissive:
EmissivePhotoSet.Empty();
break;
case ERenderCaptureType::CombinedMRS:
PackedMRSPhotoSet.Empty();
break;
case ERenderCaptureType::Opacity:
OpacityPhotoSet.Empty();
break;
case ERenderCaptureType::SubsurfaceColor:
SubsurfaceColorPhotoSet.Empty();
break;
case ERenderCaptureType::DeviceDepth:
// For the device depth photo set we have two containers to empty
DeviceDepthPhotoSet.Empty();
PhotoViewMatricies.Empty();
break;
default:
ensure(false);
}
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
TArray<FSpatialPhotoParams> UE::Geometry::ComputeStandardExteriorSpatialPhotoParameters(
UWorld* World,
const TArray<AActor*>& Actors,
FImageDimensions PhotoDimensions,
double HorizontalFOVDegrees,
double NearPlaneDist,
bool bFaces,
bool bUpperCorners,
bool bLowerCorners,
bool bUpperEdges,
bool bSideEdges)
{
if (!World || Actors.IsEmpty())
{
return {};
}
TArray<FVector3d> Directions;
if (bFaces)
{
Directions.Add(FVector3d::UnitX());
Directions.Add(-FVector3d::UnitX());
Directions.Add(FVector3d::UnitY());
Directions.Add(-FVector3d::UnitY());
Directions.Add(FVector3d::UnitZ());
Directions.Add(-FVector3d::UnitZ());
}
if (bUpperCorners)
{
Directions.Add(Normalized(FVector3d(1, 1, -1)));
Directions.Add(Normalized(FVector3d(-1, 1, -1)));
Directions.Add(Normalized(FVector3d(1, -1, -1)));
Directions.Add(Normalized(FVector3d(-1, -1, -1)));
}
if (bLowerCorners)
{
Directions.Add(Normalized(FVector3d(1, 1, 1)));
Directions.Add(Normalized(FVector3d(-1, 1, 1)));
Directions.Add(Normalized(FVector3d(1, -1, 1)));
Directions.Add(Normalized(FVector3d(-1, -1, 1)));
}
if (bUpperEdges)
{
Directions.Add(Normalized(FVector3d(-1, 0, -1)));
Directions.Add(Normalized(FVector3d(1, 0, -1)));
Directions.Add(Normalized(FVector3d(0, -1, -1)));
Directions.Add(Normalized(FVector3d(0, 1, -1)));
}
// TODO We are missing bLowerEdges!
if (bSideEdges)
{
Directions.Add(Normalized(FVector3d(1, 1, 0)));
Directions.Add(Normalized(FVector3d(-1, 1, 0)));
Directions.Add(Normalized(FVector3d(1, -1, 0)));
Directions.Add(Normalized(FVector3d(-1, -1, 0)));
}
// Compute a sphere bounding the give actors so we can use it to position the render capture viewpoints
// Note: We use FWorldRenderCapture to do this but we are not going to render anything in this function
FSphere RenderSphere;
{
FWorldRenderCapture RenderCapture;
RenderCapture.SetVisibleActors(Actors);
RenderSphere = RenderCapture.ComputeContainingRenderSphere(HorizontalFOVDegrees);
}
TArray<FSpatialPhotoParams> Result;
int32 NumDirections = Directions.Num();
for (int32 di = 0; di < NumDirections; ++di)
{
FVector3d ViewDirection = Directions[di];
ViewDirection.Normalize();
FSpatialPhotoParams Params;
Params.NearPlaneDist = NearPlaneDist;
Params.HorzFOVDegrees = HorizontalFOVDegrees;
Params.Dimensions = PhotoDimensions;
// TODO Align the frame with the renderer coordinate system then remove the axis swapping in WorldRenderCapture.cpp
Params.Frame.AlignAxis(0, ViewDirection);
Params.Frame.ConstrainedAlignAxis(2, FVector3d::UnitZ(), Params.Frame.X());
Params.Frame.Origin = RenderSphere.Center;
Params.Frame.Origin -= RenderSphere.W * Params.Frame.X();
Result.Add(Params);
}
return Result;
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
#undef LOCTEXT_NAMESPACE
Reference in New Issue Copy Permalink