izzy/UnrealEngineUWP
0
0
Fork 0
mirror of https://github.com/izzy2lost/UnrealEngineUWP.git synced 2026-03-26 18:15:20 -07:00
Code Issues Packages Projects Releases Wiki Activity
Files
00efdf60976d957eafeb1949031c62b68ec92aec
UnrealEngineUWP/Engine/Plugins/Runtime/MeshModelingToolset/Source/ModelingComponents/Private/Scene/SceneCapturePhotoSet.cpp

1030 lines
33 KiB
C++
Raw Normal View History

ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
// Copyright Epic Games, Inc. All Rights Reserved.
#include "Scene/SceneCapturePhotoSet.h"
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
#include "EngineUtils.h"
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
#include "Misc/ScopedSlowTask.h"
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
using namespace UE::Geometry;
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
#define LOCTEXT_NAMESPACE "SceneCapture"
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FSceneCapturePhotoSet::FSceneCapturePhotoSet()
{
BaseColorConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::BaseColor);
WorldNormalConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::WorldNormal);
RoughnessConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::Roughness);
MetallicConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::Metallic);
SpecularConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::Specular);
EmissiveConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::Emissive);
PackedMRSConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::CombinedMRS);
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
OpacityConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::Opacity);
SubsurfaceColorConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::SubsurfaceColor);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
DeviceDepthConfig = GetDefaultRenderCaptureConfig(ERenderCaptureType::DeviceDepth);
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
void FSceneCapturePhotoSet::SetCaptureSceneActors(UWorld* World, const TArray<AActor*>& Actors)
{
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
if (this->TargetWorld != World || this->VisibleActors != Actors)
{
// Empty the photo sets because they rendered different actors
EmptyAllPhotoSets();
// Empty the spatial photo parameters because these are computed from the bounding box of the actors
PhotoSetParams.Empty();
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
this->TargetWorld = World;
this->VisibleActors = Actors;
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
TArray<AActor*> FSceneCapturePhotoSet::GetCaptureSceneActors()
{
return VisibleActors;
}
UWorld* FSceneCapturePhotoSet::GetCaptureTargetWorld()
{
return TargetWorld;
}
void FSceneCapturePhotoSet::SetSpatialPhotoParams(const TArray<FSpatialPhotoParams>& SpatialParams)
{
// TODO Discard/reset on a per array element level rather than discarding everything when any viewpoint changed
if (PhotoSetParams != SpatialParams)
{
// Empty the photo sets because they were rendered with different viewpoints
EmptyAllPhotoSets();
PhotoSetParams = SpatialParams;
}
}
const TArray<FSpatialPhotoParams>& FSceneCapturePhotoSet::GetSpatialPhotoParams() const
{
return PhotoSetParams;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
void FSceneCapturePhotoSet::SetCaptureConfig(ERenderCaptureType CaptureType, const FRenderCaptureConfig& Config)
{
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
auto UpdateCaptureConfig1f = [Config](FRenderCaptureConfig& CaptureConfig, FSpatialPhotoSet1f& PhotoSet)
{
if (CaptureConfig.bAntiAliasing != Config.bAntiAliasing)
{
// If we're changing the AntiAliasing state we need to remove any existing photos
PhotoSet.Empty();
}
CaptureConfig = Config;
};
auto UpdateCaptureConfig3f = [Config](FRenderCaptureConfig& CaptureConfig, FSpatialPhotoSet3f& PhotoSet)
{
if (CaptureConfig.bAntiAliasing != Config.bAntiAliasing)
{
// If we're changing the AntiAliasing state we need to remove any existing photos
PhotoSet.Empty();
}
CaptureConfig = Config;
};
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureConfig3f(BaseColorConfig, BaseColorPhotoSet);
break;
case ERenderCaptureType::WorldNormal:
UpdateCaptureConfig3f(WorldNormalConfig, WorldNormalPhotoSet);
break;
case ERenderCaptureType::Roughness:
UpdateCaptureConfig1f(RoughnessConfig, RoughnessPhotoSet);
break;
case ERenderCaptureType::Metallic:
UpdateCaptureConfig1f(MetallicConfig, MetallicPhotoSet);
break;
case ERenderCaptureType::Specular:
UpdateCaptureConfig1f(SpecularConfig, SpecularPhotoSet);
break;
case ERenderCaptureType::Emissive:
UpdateCaptureConfig3f(EmissiveConfig, EmissivePhotoSet);
break;
case ERenderCaptureType::CombinedMRS:
UpdateCaptureConfig3f(PackedMRSConfig, PackedMRSPhotoSet);
break;
case ERenderCaptureType::Opacity:
UpdateCaptureConfig1f(OpacityConfig, OpacityPhotoSet);
break;
case ERenderCaptureType::SubsurfaceColor:
UpdateCaptureConfig3f(SubsurfaceColorConfig, SubsurfaceColorPhotoSet);
break;
case ERenderCaptureType::DeviceDepth:
if (DeviceDepthConfig.bAntiAliasing != Config.bAntiAliasing)
{
// If we're disabling DeviceDepth we need to remove any existing photos as well as the cached view matrices
DeviceDepthPhotoSet.Empty();
PhotoViewMatricies.Empty();
}
DeviceDepthConfig = Config;
break;
default:
ensure(false);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
}
}
FRenderCaptureConfig FSceneCapturePhotoSet::GetCaptureConfig(ERenderCaptureType CaptureType) const
{
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
return BaseColorConfig;
case ERenderCaptureType::WorldNormal:
return WorldNormalConfig;
case ERenderCaptureType::Roughness:
return RoughnessConfig;
case ERenderCaptureType::Metallic:
return MetallicConfig;
case ERenderCaptureType::Specular:
return SpecularConfig;
case ERenderCaptureType::Emissive:
return EmissiveConfig;
case ERenderCaptureType::CombinedMRS:
return PackedMRSConfig;
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
case ERenderCaptureType::Opacity:
return OpacityConfig;
case ERenderCaptureType::SubsurfaceColor:
return SubsurfaceColorConfig;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
case ERenderCaptureType::DeviceDepth:
return DeviceDepthConfig;
default:
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
ensure(false);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
}
return BaseColorConfig;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
void FSceneCapturePhotoSet::DisableAllCaptureTypes()
{
SetCaptureTypeEnabled(ERenderCaptureType::BaseColor, false);
SetCaptureTypeEnabled(ERenderCaptureType::WorldNormal, false);
SetCaptureTypeEnabled(ERenderCaptureType::Roughness, false);
SetCaptureTypeEnabled(ERenderCaptureType::Metallic, false);
SetCaptureTypeEnabled(ERenderCaptureType::Specular, false);
SetCaptureTypeEnabled(ERenderCaptureType::Emissive, false);
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
SetCaptureTypeEnabled(ERenderCaptureType::Opacity, false);
SetCaptureTypeEnabled(ERenderCaptureType::SubsurfaceColor, false);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
SetCaptureTypeEnabled(ERenderCaptureType::CombinedMRS, false);
SetCaptureTypeEnabled(ERenderCaptureType::DeviceDepth, false);
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
void FSceneCapturePhotoSet::SetCaptureTypeEnabled(ERenderCaptureType CaptureType, bool bEnabled)
{
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
auto UpdateCaptureType1f = [bEnabled](bool& bCaptureEnabled, FSpatialPhotoSet1f& PhotoSet)
{
if (!bEnabled)
{
// If we're disabling a CaptureType we need to remove any existing photos
PhotoSet.Empty();
}
bCaptureEnabled = bEnabled;
};
auto UpdateCaptureType3f = [bEnabled](bool& bCaptureEnabled, FSpatialPhotoSet3f& PhotoSet)
{
if (!bEnabled)
{
// If we're disabling a CaptureType we need to remove any existing photos
PhotoSet.Empty();
}
bCaptureEnabled = bEnabled;
};
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType3f(bEnableBaseColor, BaseColorPhotoSet);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
break;
case ERenderCaptureType::WorldNormal:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType3f(bEnableWorldNormal, WorldNormalPhotoSet);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
break;
case ERenderCaptureType::Roughness:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType1f(bEnableRoughness, RoughnessPhotoSet);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
break;
case ERenderCaptureType::Metallic:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType1f(bEnableMetallic, MetallicPhotoSet);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
break;
case ERenderCaptureType::Specular:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType1f(bEnableSpecular, SpecularPhotoSet);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
break;
case ERenderCaptureType::Emissive:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType3f(bEnableEmissive, EmissivePhotoSet);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
break;
case ERenderCaptureType::CombinedMRS:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType3f(bEnablePackedMRS, PackedMRSPhotoSet);
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
break;
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
case ERenderCaptureType::Opacity:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType1f(bEnableOpacity, OpacityPhotoSet);
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
break;
case ERenderCaptureType::SubsurfaceColor:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
UpdateCaptureType3f(bEnableSubsurfaceColor, SubsurfaceColorPhotoSet);
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
break;
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
case ERenderCaptureType::DeviceDepth:
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
if (bEnableDeviceDepth != bEnabled)
{
// If we're disabling DeviceDepth we need to remove any existing photos as well as the cached view matrices
DeviceDepthPhotoSet.Empty();
PhotoViewMatricies.Empty();
}
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
bEnableDeviceDepth = bEnabled;
break;
default:
ensure(false);
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
bool FSceneCapturePhotoSet::GetCaptureTypeEnabled(ERenderCaptureType CaptureType) const
{
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
return bEnableBaseColor;
case ERenderCaptureType::WorldNormal:
return bEnableWorldNormal;
case ERenderCaptureType::Roughness:
return bEnableRoughness;
case ERenderCaptureType::Metallic:
return bEnableMetallic;
case ERenderCaptureType::Specular:
return bEnableSpecular;
case ERenderCaptureType::Emissive:
return bEnableEmissive;
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
case ERenderCaptureType::Opacity:
return bEnableOpacity;
case ERenderCaptureType::SubsurfaceColor:
return bEnableSubsurfaceColor;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
case ERenderCaptureType::CombinedMRS:
return bEnablePackedMRS;
case ERenderCaptureType::DeviceDepth:
return bEnableDeviceDepth;
default:
ensure(false);
}
return false;
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
void FSceneCapturePhotoSet::Compute()
{
check(this->TargetWorld != nullptr);
Render Capture Baking: Fix a crash which occurred because the SceneCapture state was not properly updated when its computation was cancelled #jira UE-175047 #preflight 63d94591f626715201e45242 [CL 23933594 by matija kecman in ue5-main branch]
2023-01-31 14:12:46 -05:00
bWasCancelled = false;
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
FScopedSlowTask Progress(PhotoSetParams.Num(), LOCTEXT("ComputingViewpoints", "Computing Viewpoints..."));
Progress.MakeDialog(bAllowCancel);
// Unregister all components to remove unwanted proxies from the scene. This was previously the only way to "hide" nanite meshes, now optional.
TSet<AActor*> VisibleActorsSet(VisibleActors);
TArray<AActor*> ActorsToRegister;
if (bEnforceVisibilityViaUnregister)
{
for (TActorIterator<AActor> Actor(TargetWorld); Actor; ++Actor)
{
if (!VisibleActorsSet.Contains(*Actor))
{
Actor->UnregisterAllComponents();
ActorsToRegister.Add(*Actor);
}
}
}
ON_SCOPE_EXIT
{
// Workaround for Nanite scene proxies visibility
// Reregister all components we previously unregistered
for (AActor* Actor : ActorsToRegister)
{
Actor->RegisterAllComponents();
}
};
FWorldRenderCapture RenderCapture;
RenderCapture.SetWorld(TargetWorld);
RenderCapture.SetVisibleActors(VisibleActors);
if (bWriteDebugImages)
{
RenderCapture.SetEnableWriteDebugImage(true, 0, DebugImagesFolderName);
}
for (const FSpatialPhotoParams& Params : PhotoSetParams)
{
Progress.EnterProgressFrame(1.f);
if (Progress.ShouldCancel())
{
bWasCancelled = true;
return;
}
RenderCapture.SetDimensions(Params.Dimensions);
auto CaptureImageTypeFunc_3f = [this, &Progress, &RenderCapture, &Params]
(ERenderCaptureType CaptureType, FSpatialPhotoSet3f& PhotoSet)
{
if (PhotoSet.Num() < PhotoSetParams.Num())
{
FSpatialPhoto3f NewPhoto;
NewPhoto.Frame = Params.Frame;
NewPhoto.NearPlaneDist = Params.NearPlaneDist;
NewPhoto.HorzFOVDegrees = Params.HorzFOVDegrees;
NewPhoto.Dimensions = Params.Dimensions;
// TODO Do something with the success boolean returned by RenderCapture.CaptureFromPosition
FImageAdapter Image(&NewPhoto.Image);
FRenderCaptureConfig Config = GetCaptureConfig(CaptureType);
RenderCapture.CaptureFromPosition(CaptureType, NewPhoto.Frame, NewPhoto.HorzFOVDegrees, NewPhoto.NearPlaneDist, Image, Config);
PhotoSet.Add(MoveTemp(NewPhoto));
}
Progress.TickProgress();
};
auto CaptureImageTypeFunc_1f = [this, &Progress, &RenderCapture, &Params]
(ERenderCaptureType CaptureType, FSpatialPhotoSet1f& PhotoSet)
{
// Testing NumDirections is how we currently determine if we need to compute this photo
if (PhotoSet.Num() < PhotoSetParams.Num())
{
FSpatialPhoto1f NewPhoto;
NewPhoto.Frame = Params.Frame;
NewPhoto.NearPlaneDist = Params.NearPlaneDist;
NewPhoto.HorzFOVDegrees = Params.HorzFOVDegrees;
NewPhoto.Dimensions = Params.Dimensions;
// TODO Do something with the success boolean returned by RenderCapture.CaptureFromPosition
FImageAdapter Image(&NewPhoto.Image);
FRenderCaptureConfig Config = GetCaptureConfig(CaptureType);
RenderCapture.CaptureFromPosition(CaptureType, NewPhoto.Frame, NewPhoto.HorzFOVDegrees, NewPhoto.NearPlaneDist, Image, Config);
PhotoSet.Add(MoveTemp(NewPhoto));
}
Progress.TickProgress();
};
if (bEnableDeviceDepth)
{
CaptureImageTypeFunc_1f(ERenderCaptureType::DeviceDepth, DeviceDepthPhotoSet);
if (PhotoViewMatricies.Num() < PhotoSetParams.Num())
{
PhotoViewMatricies.Add(RenderCapture.GetLastCaptureViewMatrices());
}
}
if (bEnableBaseColor)
{
CaptureImageTypeFunc_3f(ERenderCaptureType::BaseColor, BaseColorPhotoSet);
}
if (bEnableRoughness)
{
CaptureImageTypeFunc_1f(ERenderCaptureType::Roughness, RoughnessPhotoSet);
}
if (bEnableSpecular)
{
CaptureImageTypeFunc_1f(ERenderCaptureType::Specular, SpecularPhotoSet);
}
if (bEnableMetallic)
{
CaptureImageTypeFunc_1f(ERenderCaptureType::Metallic, MetallicPhotoSet);
}
if (bEnablePackedMRS)
{
CaptureImageTypeFunc_3f(ERenderCaptureType::CombinedMRS, PackedMRSPhotoSet);
}
if (bEnableWorldNormal)
{
CaptureImageTypeFunc_3f(ERenderCaptureType::WorldNormal, WorldNormalPhotoSet);
}
if (bEnableEmissive)
{
CaptureImageTypeFunc_3f(ERenderCaptureType::Emissive, EmissivePhotoSet);
}
if (bEnableOpacity)
{
CaptureImageTypeFunc_1f(ERenderCaptureType::Opacity, OpacityPhotoSet);
}
if (bEnableSubsurfaceColor)
{
CaptureImageTypeFunc_3f(ERenderCaptureType::SubsurfaceColor, SubsurfaceColorPhotoSet);
}
} // end directions loop
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
void FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(
FImageDimensions PhotoDimensions,
double HorizontalFOVDegrees,
double NearPlaneDist,
bool bFaces,
bool bUpperCorners,
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
bool bLowerCorners,
bool bUpperEdges,
bool bSideEdges)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
TArray<FVector3d> Directions;
if (bFaces)
{
Directions.Add(FVector3d::UnitX());
Directions.Add(-FVector3d::UnitX());
Directions.Add(FVector3d::UnitY());
Directions.Add(-FVector3d::UnitY());
Directions.Add(FVector3d::UnitZ());
Directions.Add(-FVector3d::UnitZ());
}
if (bUpperCorners)
{
Directions.Add(Normalized(FVector3d(1, 1, -1)));
Directions.Add(Normalized(FVector3d(-1, 1, -1)));
Directions.Add(Normalized(FVector3d(1, -1, -1)));
Directions.Add(Normalized(FVector3d(-1, -1, -1)));
}
if (bLowerCorners)
{
Directions.Add(Normalized(FVector3d(1, 1, 1)));
Directions.Add(Normalized(FVector3d(-1, 1, 1)));
Directions.Add(Normalized(FVector3d(1, -1, 1)));
Directions.Add(Normalized(FVector3d(-1, -1, 1)));
}
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
if (bUpperEdges)
{
Directions.Add(Normalized(FVector3d(-1, 0, -1)));
Directions.Add(Normalized(FVector3d(1, 0, -1)));
Directions.Add(Normalized(FVector3d(0, -1, -1)));
Directions.Add(Normalized(FVector3d(0, 1, -1)));
}
if (bSideEdges)
{
Directions.Add(Normalized(FVector3d(1, 1, 0)));
Directions.Add(Normalized(FVector3d(-1, 1, 0)));
Directions.Add(Normalized(FVector3d(1, -1, 0)));
Directions.Add(Normalized(FVector3d(-1, -1, 0)));
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
PRAGMA_DISABLE_DEPRECATION_WARNINGS
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
AddExteriorCaptures(PhotoDimensions, HorizontalFOVDegrees, NearPlaneDist, Directions);
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
PRAGMA_ENABLE_DEPRECATION_WARNINGS
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
void FSceneCapturePhotoSet::AddExteriorCaptures(
FImageDimensions PhotoDimensions,
double HorizontalFOVDegrees,
double NearPlaneDist,
const TArray<FVector3d>& Directions)
{
check(this->TargetWorld != nullptr);
Render Capture Baking: Fix a crash which occurred because the SceneCapture state was not properly updated when its computation was cancelled #jira UE-175047 #preflight 63d94591f626715201e45242 [CL 23933594 by matija kecman in ue5-main branch]
2023-01-31 14:12:46 -05:00
bWasCancelled = false;
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
FScopedSlowTask Progress(Directions.Num(), LOCTEXT("ComputingViewpoints", "Computing Viewpoints..."));
Progress.MakeDialog(bAllowCancel);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
// Unregister all components to remove unwanted proxies from the scene. This was previously the only way to "hide" nanite meshes, now optional.
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
TSet<AActor*> VisibleActorsSet(VisibleActors);
TArray<AActor*> ActorsToRegister;
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
if (bEnforceVisibilityViaUnregister)
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
{
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
for (TActorIterator<AActor> Actor(TargetWorld); Actor; ++Actor)
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
{
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
if (!VisibleActorsSet.Contains(*Actor))
{
Actor->UnregisterAllComponents();
ActorsToRegister.Add(*Actor);
}
SceneCapturePhotoSet - "Hide" Nanite actors by temporarily unregistering components [FYI] ryan.schmidt #ROBOMERGE-SOURCE: CL 16357093 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v804-16311228) [CL 16368648 by sebastien lussier in ue5-main branch]
2021-05-18 11:54:40 -04:00
}
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
ON_SCOPE_EXIT
{
// Workaround for Nanite scene proxies visibility
// Reregister all components we previously unregistered
for (AActor* Actor : ActorsToRegister)
{
Actor->RegisterAllComponents();
}
};
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
FWorldRenderCapture RenderCapture;
RenderCapture.SetWorld(TargetWorld);
RenderCapture.SetVisibleActors(VisibleActors);
RenderCapture.SetDimensions(PhotoDimensions);
GeometryProcessing: - extend FImageAdapter to support reading ModelingComponents:: - Add AssetUtils::ForceVirtualTexturePrefetch(), ::SaveDebugImage() variants to Texture2DUtil - Add support for debug image writing to WorldRenderCapture and SceneCapturePhotoSet - FWorldRenderCapture now computes more accurate bounds (visible Components instead of Actors), and does an explicit VirtualTexture prefetch before doing render capture - FStaticMeshAssetOptions now exposes Nanite settings to allow UStaticMeshes to be created with Nanite already enabled (currently NaniteSettings.PositionPrecision defaults to max) ApproximateActors: - IGeometryProcessing_ApproximateActors::FOptions now has settings for enabling Nanite on generated meshes - ApproximateActorsImpl applies these settings, also enables VT on generated textures if necessary - FMeshApproximationSettings/FMeshApproximationTool updated w/ new Nanite settings #rb none #rnx #jira none [FYI] sebastien.lussier #preflight 60b08323f51cd90001775b98 #ROBOMERGE-OWNER: ryan.schmidt #ROBOMERGE-AUTHOR: ryan.schmidt #ROBOMERGE-SOURCE: CL 16499613 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v823-16466674) #ROBOMERGE-CONFLICT from-shelf [CL 16499647 by ryan schmidt in ue5-main branch]
2021-05-28 02:46:59 -04:00
if (bWriteDebugImages)
{
RenderCapture.SetEnableWriteDebugImage(true, 0, DebugImagesFolderName);
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
// this tells us origin and radius - could be view-dependent...
FSphere RenderSphere = RenderCapture.ComputeContainingRenderSphere(HorizontalFOVDegrees);
int32 NumDirections = Directions.Num();
for (int32 di = 0; di < NumDirections; ++di)
{
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
Progress.EnterProgressFrame(1.f);
if (Progress.ShouldCancel())
{
bWasCancelled = true;
return;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
FVector3d ViewDirection = Directions[di];
ViewDirection.Normalize();
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
FSpatialPhotoParams Params;
Params.NearPlaneDist = NearPlaneDist;
Params.HorzFOVDegrees = HorizontalFOVDegrees;
Params.Dimensions = PhotoDimensions;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
// TODO Align the frame with the renderer coordinate system then remove the axis swapping in WorldRenderCapture.cpp
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
Params.Frame.AlignAxis(0, ViewDirection);
Params.Frame.ConstrainedAlignAxis(2, FVector3d::UnitZ(), Params.Frame.X());
Params.Frame.Origin = RenderSphere.Center;
Params.Frame.Origin -= RenderSphere.W * Params.Frame.X();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
auto CaptureImageTypeFunc_3f = [this, &Progress, &RenderCapture, &Params, &NumDirections]
(ERenderCaptureType CaptureType, FSpatialPhotoSet3f& PhotoSet)
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
// Test NumDirections to avoid recomputing photo sets. Search :SceneCaptureWithExistingCaptures
if (PhotoSet.Num() < NumDirections)
{
FSpatialPhoto3f NewPhoto;
NewPhoto.Frame = Params.Frame;
NewPhoto.NearPlaneDist = Params.NearPlaneDist;
NewPhoto.HorzFOVDegrees = Params.HorzFOVDegrees;
NewPhoto.Dimensions = Params.Dimensions;
// TODO Do something with the success boolean returned by RenderCapture.CaptureFromPosition
FImageAdapter Image(&NewPhoto.Image);
FRenderCaptureConfig Config = GetCaptureConfig(CaptureType);
RenderCapture.CaptureFromPosition(CaptureType, NewPhoto.Frame, NewPhoto.HorzFOVDegrees, NewPhoto.NearPlaneDist, Image, Config);
PhotoSet.Add(MoveTemp(NewPhoto));
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
Progress.TickProgress();
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
};
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
auto CaptureImageTypeFunc_1f = [this, &Progress, &RenderCapture, &Params, &NumDirections]
(ERenderCaptureType CaptureType, FSpatialPhotoSet1f& PhotoSet)
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
{
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
// Test NumDirections to avoid recomputing photo sets. Search :SceneCaptureWithExistingCaptures
if (PhotoSet.Num() < NumDirections)
{
FSpatialPhoto1f NewPhoto;
NewPhoto.Frame = Params.Frame;
NewPhoto.NearPlaneDist = Params.NearPlaneDist;
NewPhoto.HorzFOVDegrees = Params.HorzFOVDegrees;
NewPhoto.Dimensions = Params.Dimensions;
// TODO Do something with the success boolean returned by RenderCapture.CaptureFromPosition
FImageAdapter Image(&NewPhoto.Image);
FRenderCaptureConfig Config = GetCaptureConfig(CaptureType);
RenderCapture.CaptureFromPosition(CaptureType, NewPhoto.Frame, NewPhoto.HorzFOVDegrees, NewPhoto.NearPlaneDist, Image, Config);
PhotoSet.Add(MoveTemp(NewPhoto));
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
Progress.TickProgress();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
};
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
if (bEnableDeviceDepth)
{
CaptureImageTypeFunc_1f(ERenderCaptureType::DeviceDepth, DeviceDepthPhotoSet);
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
if (PhotoViewMatricies.Num() < NumDirections)
{
PhotoViewMatricies.Add(RenderCapture.GetLastCaptureViewMatrices());
}
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
if (bEnableBaseColor)
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_3f(ERenderCaptureType::BaseColor, BaseColorPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
if (bEnableRoughness)
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_1f(ERenderCaptureType::Roughness, RoughnessPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
if (bEnableSpecular)
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_1f(ERenderCaptureType::Specular, SpecularPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
if (bEnableMetallic)
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_1f(ERenderCaptureType::Metallic, MetallicPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
ModelingComponents: support Packed Metallic/Roughness/Specular (MRS) capture in WorldRenderCapture using custom postprocess render pass. Also support in FSceneCapturePhotoSet. Add Linear Color variant to FTexture2DBuilder. #rb none #rnx #jira none #preflight 60ade5d11d02bb0001883281 #ROBOMERGE-SOURCE: CL 16465354 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v818-16446889) [CL 16465367 by ryan schmidt in ue5-main branch]
2021-05-26 03:02:11 -04:00
if (bEnablePackedMRS)
{
CaptureImageTypeFunc_3f(ERenderCaptureType::CombinedMRS, PackedMRSPhotoSet);
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
if (bEnableWorldNormal)
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_3f(ERenderCaptureType::WorldNormal, WorldNormalPhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
if (bEnableEmissive)
{
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
CaptureImageTypeFunc_3f(ERenderCaptureType::Emissive, EmissivePhotoSet);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
if (bEnableOpacity)
{
CaptureImageTypeFunc_1f(ERenderCaptureType::Opacity, OpacityPhotoSet);
}
if (bEnableSubsurfaceColor)
{
CaptureImageTypeFunc_3f(ERenderCaptureType::SubsurfaceColor, SubsurfaceColorPhotoSet);
}
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
// AddExteriorCaptures can be called on an FSceneCapturePhotoSet which already has some capture types computed
// and in this case we should not modify the existing photo sets/cached photo set parameters.
// See :SceneCaptureWithExistingCaptures
if (PhotoSetParams.Num() < NumDirections)
{
PhotoSetParams.Add(Params);
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
} // end directions loop
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
void FSceneCapturePhotoSet::OptimizePhotoSets()
{
// todo:
// 1) crop photos to regions with actual pixels
// 2) pack into fewer photos (eg pack spec/rough/metallic)
// 3) RLE encoding or other compression
}
FSceneCapturePhotoSet::FSceneSample::FSceneSample()
{
HaveValues = FRenderCaptureTypeFlags::None();
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
BaseColor = FVector3f(0, 0, 0);
Roughness = 0.0f;
Specular = 0.0f;
Metallic = 0.0f;
Emissive = FVector3f(0, 0, 0);
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
Opacity = 0.0f;
SubsurfaceColor = FVector3f(0, 0, 0);
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
WorldNormal = FVector3f(0, 0, 1);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
DeviceDepth = 0.0f;
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
FVector3f FSceneCapturePhotoSet::FSceneSample::GetValue3f(ERenderCaptureType CaptureType) const
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
{
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
return BaseColor;
case ERenderCaptureType::WorldNormal:
return WorldNormal;
case ERenderCaptureType::Roughness:
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
return Roughness * FVector3f::One();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
case ERenderCaptureType::Metallic:
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
return Metallic * FVector3f::One();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
case ERenderCaptureType::Specular:
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
return Specular * FVector3f::One();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
case ERenderCaptureType::Emissive:
return Emissive;
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
case ERenderCaptureType::Opacity:
return Opacity * FVector3f::One();
case ERenderCaptureType::SubsurfaceColor:
return SubsurfaceColor;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
case ERenderCaptureType::DeviceDepth:
return DeviceDepth * FVector3f::One();
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
default:
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
ensure(false);
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
}
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
return FVector3f::Zero();
}
FVector4f FSceneCapturePhotoSet::FSceneSample::GetValue4f(ERenderCaptureType CaptureType) const
{
switch (CaptureType)
{
case ERenderCaptureType::BaseColor:
return FVector4f(BaseColor.X, BaseColor.Y, BaseColor.Z, 1.0f);
case ERenderCaptureType::WorldNormal:
return FVector4f(WorldNormal.X, WorldNormal.Y, WorldNormal.Z, 1.0f);
case ERenderCaptureType::Roughness:
return FVector4f(Roughness, Roughness, Roughness, 1.0f);
case ERenderCaptureType::Metallic:
return FVector4f(Metallic, Metallic, Metallic, 1.0f);
case ERenderCaptureType::Specular:
return FVector4f(Specular, Specular, Specular, 1.0f);
ModelingComponents: support Packed Metallic/Roughness/Specular (MRS) capture in WorldRenderCapture using custom postprocess render pass. Also support in FSceneCapturePhotoSet. Add Linear Color variant to FTexture2DBuilder. #rb none #rnx #jira none #preflight 60ade5d11d02bb0001883281 #ROBOMERGE-SOURCE: CL 16465354 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v818-16446889) [CL 16465367 by ryan schmidt in ue5-main branch]
2021-05-26 03:02:11 -04:00
case ERenderCaptureType::CombinedMRS:
return FVector4f(Metallic, Roughness, Specular, 1.0f);
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
case ERenderCaptureType::Emissive:
return FVector4f(Emissive.X, Emissive.Y, Emissive.Z, 1.0f);
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
case ERenderCaptureType::Opacity:
return FVector4f(Opacity, Opacity, Opacity, 1.0f);
case ERenderCaptureType::SubsurfaceColor:
return FVector4f(SubsurfaceColor.X, SubsurfaceColor.Y, SubsurfaceColor.Z, 1.0f);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
case ERenderCaptureType::DeviceDepth:
return FVector4f(DeviceDepth, DeviceDepth, DeviceDepth, 1.0f);
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
default:
RenderCaptureBaking: Improve performance when adding/removing capture channels - When adding a new channel: Only the newly requested channel is captured, previously all enabled channels were (re)captured - When removing an already captured channel: We no longer recompute all remaining channels, we just wipe the one that was removed #rnx #rb ryan.schmidt #preflight 6372120fbf76990b71f17750 [CL 23115901 by matija kecman in ue5-main branch]
2022-11-14 05:09:53 -05:00
ensure(false);
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
return FVector4f::Zero();
}
Render Capture Baking: Move visibility raycast from evaluators to detail sampler Benchmark configuration: - Test scene with 4 meshes: a mossy stone nanite mesh, a beach ball, a ball with an emissive blue cross and a hut mesh - Target mesh: 11.8k triangles mesh, created by voxel wrapping the meshes in the test scene elements - 4096x4096 output texture with 16x multisampling baking BaseColor, Emissive, PackedMRS and Normal maps - 4096x4096 photo capture resolution Unreal Insights reports the following times for the Bake call: - Before this change: 2 mins 45 seconds - After this change: 55.9 seconds Aside: (Unaffected by this change) In both cases the photo capture, which is done on the game thread, took about 1 min #jira none #rb #rnx #preflight 627e64e01748fbc85bb24a42 [CL 20221383 by matija kecman in ue5-main branch]
2022-05-16 05:13:25 -04:00
bool FSceneCapturePhotoSet::ComputeSampleLocation(
const FVector3d& Position,
const FVector3d& Normal,
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
const float ValidSampleDepthThreshold,
Render Capture Baking: Move visibility raycast from evaluators to detail sampler Benchmark configuration: - Test scene with 4 meshes: a mossy stone nanite mesh, a beach ball, a ball with an emissive blue cross and a hut mesh - Target mesh: 11.8k triangles mesh, created by voxel wrapping the meshes in the test scene elements - 4096x4096 output texture with 16x multisampling baking BaseColor, Emissive, PackedMRS and Normal maps - 4096x4096 photo capture resolution Unreal Insights reports the following times for the Bake call: - Before this change: 2 mins 45 seconds - After this change: 55.9 seconds Aside: (Unaffected by this change) In both cases the photo capture, which is done on the game thread, took about 1 min #jira none #rb #rnx #preflight 627e64e01748fbc85bb24a42 [CL 20221383 by matija kecman in ue5-main branch]
2022-05-16 05:13:25 -04:00
TFunctionRef<bool(const FVector3d&, const FVector3d&)> VisibilityFunction,
int& PhotoIndex,
FVector2d& PhotoCoords) const
{
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
double DotTolerance = -0.1; // dot should be negative for normal pointing towards photo
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
if (ValidSampleDepthThreshold > 0)
{
check(DeviceDepthPhotoSet.Num() == PhotoSetParams.Num());
check(PhotoViewMatricies.Num() == PhotoSetParams.Num());
}
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
PhotoIndex = IndexConstants::InvalidID;
PhotoCoords = FVector2d(0., 0.);
double MinDot = 1.0;
int32 NumPhotos = PhotoSetParams.Num();
Fix Editor Mac which broke in Horde Issue #180297: Compile errors in Module.ModelingComponents.cpp and SceneCapturePhotoSet.cpp #rnx #rb jimmy.andrews #jira none #preflight 62a7511aaf7ad033effcec61 #preflight 62a7511aaf7ad033effcec61 [CL 20627748 by matija kecman in ue5-main branch]
2022-06-13 11:19:07 -04:00
for (int32 Index = 0; Index < NumPhotos; ++Index)
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
{
Fix Editor Mac which broke in Horde Issue #180297: Compile errors in Module.ModelingComponents.cpp and SceneCapturePhotoSet.cpp #rnx #rb jimmy.andrews #jira none #preflight 62a7511aaf7ad033effcec61 #preflight 62a7511aaf7ad033effcec61 [CL 20627748 by matija kecman in ue5-main branch]
2022-06-13 11:19:07 -04:00
const FSpatialPhotoParams& Params = PhotoSetParams[Index];
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
check(Params.Dimensions.IsSquare());
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FFrame3d RenderFrame(Params.Frame.Origin, Params.Frame.Y(), Params.Frame.Z(), Params.Frame.X());
FVector3d ViewDirection = RenderFrame.Z();
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
double ViewDot = ViewDirection.Dot(Normal);
if (ViewDot > DotTolerance || ViewDot > MinDot)
{
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
// The sample is facing away from the photo, or we found a photo more aligned with this sample
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
continue;
}
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FFrame3d ViewPlane = RenderFrame;
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
ViewPlane.Origin += Params.NearPlaneDist * ViewDirection;
double ViewPlaneWidthWorld = Params.NearPlaneDist * FMathd::Tan(Params.HorzFOVDegrees * 0.5 * FMathd::DegToRad);
double ViewPlaneHeightWorld = ViewPlaneWidthWorld;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
// Shoot a ray from the camera position toward the sample position and find the hit point on photo plane
constexpr int NormalAxisIndex = 2;
FVector3d RayOrigin = RenderFrame.Origin;
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
FVector3d RayDir = Normalized(Position - RayOrigin);
FVector3d HitPoint;
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
bool bHit = ViewPlane.RayPlaneIntersection(RayOrigin, RayDir, NormalAxisIndex, HitPoint);
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
if (bHit)
{
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FVector2d DeviceXY;
DeviceXY.X = (HitPoint - ViewPlane.Origin).Dot(ViewPlane.X()) / ViewPlaneWidthWorld;
DeviceXY.Y = (HitPoint - ViewPlane.Origin).Dot(ViewPlane.Y()) / ViewPlaneHeightWorld;
if (FMathd::Abs(DeviceXY.X) < 1 && FMathd::Abs(DeviceXY.Y) < 1)
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
{
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
// Shoot a ray from the sample position toward the camera position checking occlusion
bool bVisible = VisibilityFunction(Position, RayOrigin);
if (bVisible)
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
{
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
FVector2d UVCoords = FRenderCaptureCoordinateConverter2D::DeviceToUV(DeviceXY);
if (ValidSampleDepthThreshold > 0)
{
// Look up the device depth from the depth photo set, these values are from 0 (far plane) to
// 1 (near plane). We skip points which would unproject to the far plane, which is positioned at
// infinity, we also do not interpolate the depth values, doing so is not a good approximation
// of the underlying scene
float DeviceZ = DeviceDepthPhotoSet.Get(Index).Image.NearestSampleUV(UVCoords);
if (DeviceZ > 0)
{
// Compute the pixel position in world space to use it to compute a depth according to the render
FVector3d PixelPositionDevice{DeviceXY, DeviceZ};
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
FVector4d PixelPositionWorld = PhotoViewMatricies[Index].GetInvViewProjectionMatrix().TransformPosition(PixelPositionDevice);
Render Capture Baking: Use depth capture to eliminate occlusion artefacts which can appear when baking non-convex shapes and look like weird sihouettes/blotches in the base color channel Also removed artefacts that became visible after the ones fixible with a depth test were resolved. These were related to linearly interpolating data on the photosets, this was fixed since the baking framework is where texture filtering should take place #rnx #rb lonnie.li, ryan.schmidt #jira none #preflight 62d473b0dc4397d384a06bc8 [CL 21148985 by matija kecman in ue5-main branch]
2022-07-18 11:21:15 -04:00
PixelPositionWorld /= PixelPositionWorld.W;
// Compare the depth of the sample with the depth of the pixel and consider the sample invalid
// if these do not match closely enough. This fixes artefacts which occur when sample ray just
// misses an obstruction and hits a pixel where the color was set by a slightly different ray,
// through the pixel center, which does hit the obstruction. This problem occurs because the
// depth capture was obtained by renderering the the source meshes but the visiblity function
// works on the target mesh
float PixelDepth = (RayOrigin - FVector3d(PixelPositionWorld)).Length();
float SampleDepth = (RayOrigin - Position).Length();
if (FMath::Abs(PixelDepth - SampleDepth) < ValidSampleDepthThreshold)
{
PhotoCoords.X = UVCoords.X * (double)Params.Dimensions.GetWidth();
PhotoCoords.Y = UVCoords.Y * (double)Params.Dimensions.GetHeight();
PhotoIndex = Index;
MinDot = ViewDot;
}
} // Test DeviceZ > 0
}
else
{
PhotoCoords.X = UVCoords.X * (double)Params.Dimensions.GetWidth();
PhotoCoords.Y = UVCoords.Y * (double)Params.Dimensions.GetHeight();
PhotoIndex = Index;
MinDot = ViewDot;
}
} // Test bVisible
} // Test UVCoords in (-1,1)x(-1,1)
} // Hit photo plane
} // Photo loop
Render Capture Baking: Add messages warning users about missing tangents or UVs #rnx #rb lonnie.li #jira none #preflight 62a72d695a44fbc402908e21 [CL 20626847 by matija kecman in ue5-main branch]
2022-06-13 10:27:34 -04:00
return PhotoIndex != IndexConstants::InvalidID;
Render Capture Baking: Move visibility raycast from evaluators to detail sampler Benchmark configuration: - Test scene with 4 meshes: a mossy stone nanite mesh, a beach ball, a ball with an emissive blue cross and a hut mesh - Target mesh: 11.8k triangles mesh, created by voxel wrapping the meshes in the test scene elements - 4096x4096 output texture with 16x multisampling baking BaseColor, Emissive, PackedMRS and Normal maps - 4096x4096 photo capture resolution Unreal Insights reports the following times for the Bake call: - Before this change: 2 mins 45 seconds - After this change: 55.9 seconds Aside: (Unaffected by this change) In both cases the photo capture, which is done on the game thread, took about 1 min #jira none #rb #rnx #preflight 627e64e01748fbc85bb24a42 [CL 20221383 by matija kecman in ue5-main branch]
2022-05-16 05:13:25 -04:00
}
GeometryProcessing: - Add FImageAdapter, wraps TImageBuilder of different types and provides a consistent interface, so that other code doesn't need specializations for each image type - FMeshSceneAdapter now supports the ability to detect and thicken "thin" meshes, by flattening the instances into unique mesh copies (generally necessary due to scaling). Can also now return statistics about the mesh scene, return a full flattened mesh (for debugging), and automatically generate a "closing mesh" that can help to cap off the base of mesh assemblies when trying to generate a solid/etc. ModelingComponents: - FWorldRenderCapture now writes via FImageAdapter instead of an Image4f - FSceneCapturePhotoSet now stores less data for photo sets. BaseColor/Emissive/WorldNormal are 3f, Roughness/Specular/Metallic are 1f - IApproximateActors and implementation now support optional base-capping, occluded mesh removal, geometric-tolerance simplification, thin mesh auth-thickening. Now creates MICs based on an input Material, instead of new Material. Added ability to emit a flattened debug mesh, and print debug info. MergeActors: - Expose above improvements to IApproximateActors #rb none #rnx #jira none #preflight 609ef005f6c6e3000144c5e2 [CL 16338550 by Ryan Schmidt in ue5-main branch]
2021-05-14 21:11:51 -04:00
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
bool FSceneCapturePhotoSet::ComputeSample(
const FRenderCaptureTypeFlags& SampleChannels,
const FVector3d& Position,
const FVector3d& Normal,
TFunctionRef<bool(const FVector3d&, const FVector3d&)> VisibilityFunction,
FSceneSample& DefaultsInResultsOut) const
{
// This could be much more efficient if (eg) we knew that all the photo sets have
// the same captures, then the query only has to be done once and can be used to sample each specific photo
Render Capture Baking: Move visibility raycast from evaluators to detail sampler Benchmark configuration: - Test scene with 4 meshes: a mossy stone nanite mesh, a beach ball, a ball with an emissive blue cross and a hut mesh - Target mesh: 11.8k triangles mesh, created by voxel wrapping the meshes in the test scene elements - 4096x4096 output texture with 16x multisampling baking BaseColor, Emissive, PackedMRS and Normal maps - 4096x4096 photo capture resolution Unreal Insights reports the following times for the Bake call: - Before this change: 2 mins 45 seconds - After this change: 55.9 seconds Aside: (Unaffected by this change) In both cases the photo capture, which is done on the game thread, took about 1 min #jira none #rb #rnx #preflight 627e64e01748fbc85bb24a42 [CL 20221383 by matija kecman in ue5-main branch]
2022-05-16 05:13:25 -04:00
// This is implemented in the other ComputeSample overload
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
if (SampleChannels.bBaseColor)
{
DefaultsInResultsOut.BaseColor =
BaseColorPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.BaseColor);
DefaultsInResultsOut.HaveValues.bBaseColor = true;
}
if (SampleChannels.bRoughness)
{
DefaultsInResultsOut.Roughness =
RoughnessPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Roughness);
DefaultsInResultsOut.HaveValues.bRoughness = true;
}
if (SampleChannels.bSpecular)
{
DefaultsInResultsOut.Specular =
SpecularPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Specular);
DefaultsInResultsOut.HaveValues.bSpecular = true;
}
if (SampleChannels.bMetallic)
{
DefaultsInResultsOut.Metallic =
MetallicPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Metallic);
DefaultsInResultsOut.HaveValues.bMetallic = true;
}
ModelingComponents: support Packed Metallic/Roughness/Specular (MRS) capture in WorldRenderCapture using custom postprocess render pass. Also support in FSceneCapturePhotoSet. Add Linear Color variant to FTexture2DBuilder. #rb none #rnx #jira none #preflight 60ade5d11d02bb0001883281 #ROBOMERGE-SOURCE: CL 16465354 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v818-16446889) [CL 16465367 by ryan schmidt in ue5-main branch]
2021-05-26 03:02:11 -04:00
if (SampleChannels.bCombinedMRS)
{
FVector3f MRSValue(DefaultsInResultsOut.Metallic, DefaultsInResultsOut.Roughness, DefaultsInResultsOut.Specular);
MRSValue = PackedMRSPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, MRSValue);
DefaultsInResultsOut.Metallic = MRSValue.X;
DefaultsInResultsOut.Roughness = MRSValue.Y;
DefaultsInResultsOut.Specular = MRSValue.Z;
DefaultsInResultsOut.HaveValues.bMetallic = true;
DefaultsInResultsOut.HaveValues.bRoughness = true;
DefaultsInResultsOut.HaveValues.bSpecular = true;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
if (SampleChannels.bEmissive)
{
DefaultsInResultsOut.Emissive =
EmissivePhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Emissive);
DefaultsInResultsOut.HaveValues.bEmissive = true;
}
Render Capture Baking: Add SubsurfaceColor and Opacity channels #jira FORT-509766 #rb lonnie.li #preflight 63b5fbd6202bee5e272b66c3 [CL 23585842 by matija kecman in ue5-main branch]
2023-01-05 05:19:44 -05:00
if (SampleChannels.bOpacity)
{
DefaultsInResultsOut.Opacity =
OpacityPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.Opacity);
DefaultsInResultsOut.HaveValues.bOpacity = true;
}
if (SampleChannels.bSubsurfaceColor)
{
DefaultsInResultsOut.SubsurfaceColor =
SubsurfaceColorPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.SubsurfaceColor);
DefaultsInResultsOut.HaveValues.bSubsurfaceColor = true;
}
ModelingComponents: add support for capturing scene renders and doing photo-based texturing - new FWorldRenderCapture class can capture various render buffers of a set of Actors from a given 3D camera - new TSpatialPhoto represents a 3D-oriented 2D image (ie a "photo") - new TSpatialPhotoSet stores a set of photos and supports quering them for an attribute value "seen by" the photo set at an arbitrary 3D position (eg basically photo-reprojection onto the point) - new FSceneCapturePhotoSet calculates TSpatialPhotoSet for a set of render attributes using FWorldRenderCapture, and then supports querying to effectively texture a 3D point using the Photo Set - FTexture2DBuilder now knows about various other texture types, clarified SRGB conversion parameters #rb none #rnx #jira none [CL 16135423 by Ryan Schmidt in ue5-main branch]
2021-04-27 21:11:58 -04:00
if (SampleChannels.bWorldNormal)
{
DefaultsInResultsOut.WorldNormal =
WorldNormalPhotoSet.ComputeSample(Position, Normal, VisibilityFunction, DefaultsInResultsOut.WorldNormal);
DefaultsInResultsOut.HaveValues.bWorldNormal = true;
}
return true;
GeometryProcessing: - extend FImageAdapter to support reading ModelingComponents:: - Add AssetUtils::ForceVirtualTexturePrefetch(), ::SaveDebugImage() variants to Texture2DUtil - Add support for debug image writing to WorldRenderCapture and SceneCapturePhotoSet - FWorldRenderCapture now computes more accurate bounds (visible Components instead of Actors), and does an explicit VirtualTexture prefetch before doing render capture - FStaticMeshAssetOptions now exposes Nanite settings to allow UStaticMeshes to be created with Nanite already enabled (currently NaniteSettings.PositionPrecision defaults to max) ApproximateActors: - IGeometryProcessing_ApproximateActors::FOptions now has settings for enabling Nanite on generated meshes - ApproximateActorsImpl applies these settings, also enables VT on generated textures if necessary - FMeshApproximationSettings/FMeshApproximationTool updated w/ new Nanite settings #rb none #rnx #jira none [FYI] sebastien.lussier #preflight 60b08323f51cd90001775b98 #ROBOMERGE-OWNER: ryan.schmidt #ROBOMERGE-AUTHOR: ryan.schmidt #ROBOMERGE-SOURCE: CL 16499613 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v823-16466674) #ROBOMERGE-CONFLICT from-shelf [CL 16499647 by ryan schmidt in ue5-main branch]
2021-05-28 02:46:59 -04:00
}
Add additional capture locations to FSceneCapturePhotoSet::AddStandardExteriorCapturesFromBoundingBox(), on upper and side box edge midpoints. Enable by default in Approximate Actors and BakeRenderCaptureTool. #rb jimmy.andrews #preflight 62a11612e820a589df5e36dd #fyi matija.kecman [CL 20570945 by Ryan Schmidt in ue5-main branch]
2022-06-08 23:39:09 -04:00
void FSceneCapturePhotoSet::SetEnableVisibilityByUnregisterMode(bool bEnable)
{
bEnforceVisibilityViaUnregister = bEnable;
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
void FSceneCapturePhotoSet::EmptyAllPhotoSets()
{
BaseColorPhotoSet.Empty();
RoughnessPhotoSet.Empty();
SpecularPhotoSet.Empty();
MetallicPhotoSet.Empty();
PackedMRSPhotoSet.Empty();
WorldNormalPhotoSet.Empty();
EmissivePhotoSet.Empty();
OpacityPhotoSet.Empty();
SubsurfaceColorPhotoSet.Empty();
// For the device depth photo set we have two containers to empty
DeviceDepthPhotoSet.Empty();
PhotoViewMatricies.Empty();
}
GeometryProcessing: - extend FImageAdapter to support reading ModelingComponents:: - Add AssetUtils::ForceVirtualTexturePrefetch(), ::SaveDebugImage() variants to Texture2DUtil - Add support for debug image writing to WorldRenderCapture and SceneCapturePhotoSet - FWorldRenderCapture now computes more accurate bounds (visible Components instead of Actors), and does an explicit VirtualTexture prefetch before doing render capture - FStaticMeshAssetOptions now exposes Nanite settings to allow UStaticMeshes to be created with Nanite already enabled (currently NaniteSettings.PositionPrecision defaults to max) ApproximateActors: - IGeometryProcessing_ApproximateActors::FOptions now has settings for enabling Nanite on generated meshes - ApproximateActorsImpl applies these settings, also enables VT on generated textures if necessary - FMeshApproximationSettings/FMeshApproximationTool updated w/ new Nanite settings #rb none #rnx #jira none [FYI] sebastien.lussier #preflight 60b08323f51cd90001775b98 #ROBOMERGE-OWNER: ryan.schmidt #ROBOMERGE-AUTHOR: ryan.schmidt #ROBOMERGE-SOURCE: CL 16499613 in //UE5/Private-Frosty/... #ROBOMERGE-BOT: STARSHIP (Private-Frosty -> Main) (v823-16466674) #ROBOMERGE-CONFLICT from-shelf [CL 16499647 by ryan schmidt in ue5-main branch]
2021-05-28 02:46:59 -04:00
void FSceneCapturePhotoSet::SetEnableWriteDebugImages(bool bEnable, FString FolderName)
{
bWriteDebugImages = bEnable;
if (FolderName.Len() > 0)
{
DebugImagesFolderName = FolderName;
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
}
Render Capture Baking: Refactor how the scene capture and baking steps are managed to fix a crash when scene capture parameters changed and implement other improvements The crash that was fixed occured when, for example, the userturned off a capture channel while the baking background compute was running The refactoring has enabled the tool to only bake the newly captured channels only, previously when the scene capture was incrementally updated all pre-existing channels were baked again The refactoring should enable us to more easily support a feature where the user toggles capture directions or adds new ones manually #preflight 63c02c46345a532dfc273e6a #rb lonnie.li #rnx [CL 23663959 by matija kecman in ue5-main branch]
2023-01-12 11:09:29 -05:00
TArray<FSpatialPhotoParams> UE::Geometry::ComputeStandardExteriorSpatialPhotoParameters(
UWorld* World,
const TArray<AActor*>& Actors,
FImageDimensions PhotoDimensions,
double HorizontalFOVDegrees,
double NearPlaneDist,
bool bFaces,
bool bUpperCorners,
bool bLowerCorners,
bool bUpperEdges,
bool bSideEdges)
{
if (!World || Actors.IsEmpty())
{
return {};
}
TArray<FVector3d> Directions;
if (bFaces)
{
Directions.Add(FVector3d::UnitX());
Directions.Add(-FVector3d::UnitX());
Directions.Add(FVector3d::UnitY());
Directions.Add(-FVector3d::UnitY());
Directions.Add(FVector3d::UnitZ());
Directions.Add(-FVector3d::UnitZ());
}
if (bUpperCorners)
{
Directions.Add(Normalized(FVector3d(1, 1, -1)));
Directions.Add(Normalized(FVector3d(-1, 1, -1)));
Directions.Add(Normalized(FVector3d(1, -1, -1)));
Directions.Add(Normalized(FVector3d(-1, -1, -1)));
}
if (bLowerCorners)
{
Directions.Add(Normalized(FVector3d(1, 1, 1)));
Directions.Add(Normalized(FVector3d(-1, 1, 1)));
Directions.Add(Normalized(FVector3d(1, -1, 1)));
Directions.Add(Normalized(FVector3d(-1, -1, 1)));
}
if (bUpperEdges)
{
Directions.Add(Normalized(FVector3d(-1, 0, -1)));
Directions.Add(Normalized(FVector3d(1, 0, -1)));
Directions.Add(Normalized(FVector3d(0, -1, -1)));
Directions.Add(Normalized(FVector3d(0, 1, -1)));
}
// TODO We are missing bLowerEdges!
if (bSideEdges)
{
Directions.Add(Normalized(FVector3d(1, 1, 0)));
Directions.Add(Normalized(FVector3d(-1, 1, 0)));
Directions.Add(Normalized(FVector3d(1, -1, 0)));
Directions.Add(Normalized(FVector3d(-1, -1, 0)));
}
// Compute a sphere bounding the give actors so we can use it to position the render capture viewpoints
// Note: We use FWorldRenderCapture to do this but we are not going to render anything in this function
FSphere RenderSphere;
{
FWorldRenderCapture RenderCapture;
RenderCapture.SetVisibleActors(Actors);
RenderSphere = RenderCapture.ComputeContainingRenderSphere(HorizontalFOVDegrees);
}
TArray<FSpatialPhotoParams> Result;
int32 NumDirections = Directions.Num();
for (int32 di = 0; di < NumDirections; ++di)
{
FVector3d ViewDirection = Directions[di];
ViewDirection.Normalize();
FSpatialPhotoParams Params;
Params.NearPlaneDist = NearPlaneDist;
Params.HorzFOVDegrees = HorizontalFOVDegrees;
Params.Dimensions = PhotoDimensions;
// TODO Align the frame with the renderer coordinate system then remove the axis swapping in WorldRenderCapture.cpp
Params.Frame.AlignAxis(0, ViewDirection);
Params.Frame.ConstrainedAlignAxis(2, FVector3d::UnitZ(), Params.Frame.X());
Params.Frame.Origin = RenderSphere.Center;
Params.Frame.Origin -= RenderSphere.W * Params.Frame.X();
Result.Add(Params);
}
return Result;
}
Render Capture Baking: Add FScopedSlowTask to give user progress feedback on game thread scene capture #rb lonnie.li #rnx #jira none #preflight 62822298046b81bf9399584e [CL 20221812 by matija kecman in ue5-main branch]
2022-05-16 06:50:35 -04:00
#undef LOCTEXT_NAMESPACE
Reference in New Issue Copy Permalink