69 Commits

Author SHA1 Message Date
sean sweeney
40656279aa FunctionalTest tagging and filtering
Exposes a "Test Tags" field to the Editor UI for Actor objects implementing FunctionalTest, such as BlueprintFunctionalTest and ScreenshotFunctionalTest. This persists user-defined tag data via the asset cache, and supports hot-editing.

#jira UE-213249
#rb jerome.delattre, rob.huyett, sebastian.lewicki

[CL 35829246 by sean sweeney in ue5-main branch]
2024-08-27 12:52:26 -04:00
sean sweeney
e995cbd272 [Backout] CL 35802356
Resolves breaking change on Mac which needs investigation

#jira UE-213249

[CL 35803184 by sean sweeney in ue5-main branch]
2024-08-26 13:18:36 -04:00
sean sweeney
4219762f70 FunctionalTest tagging and filtering
Exposes a "Test Tags" field to the Editor UI for Actor objects implementing FunctionalTest, such as BlueprintFunctionalTest and ScreenshotFunctionalTest. This persists user-defined tag data via the asset cache, and supports hot-editing.

#jira UE-213249
#rb jerome.delattre, rob.huyett, sebastian.lewicki

[CL 35802364 by sean sweeney in ue5-main branch]
2024-08-26 12:46:27 -04:00
sebastian lewicki
9caf645cd5 Fixing crash when saving an Actor to a GroundTruthData asset for test validation as the Actor does not have a valid level associated with it.
#jira UE-216179
#jira UE-216180
#rnx
#rb aditya.ravichandran, Jerome.Delattre


#virtualized

[CL 34151709 by sebastian lewicki in ue5-main branch]
2024-06-06 06:08:34 -04:00
oleg krasnov
99d23a4344 FunctionalTesting - Make screenshot test actor details more intuitive while working with DisableNoisyRenderingFeatures/EyeAdaptation options.
#jira UE-117952
#rb Jerome.Delattre
#rnx

[CL 30175002 by oleg krasnov in ue5-main branch]
2023-12-07 02:32:18 -05:00
sergei svetlov
dc071d7436 Support the capability to add info, warning and error directly from Python test
#jira UE-198836
#rb jerome.delattre, sebastian.lewicki-qa-sdets

[CL 29359650 by sergei svetlov in ue5-main branch]
2023-11-02 06:07:56 -04:00
Jerome Delattre
50a2846584 Avoid Take High Res Screenshot scripting command to unset viewport user settings
#jira PLAY-12210
#rnx
#rb Rob.Huyett, Sean.Sweeney

[CL 27966880 by Jerome Delattre in ue5-main branch]
2023-09-18 14:02:00 -04:00
Jerome Delattre
31881299d9 Add support for plain text to FAutomationTestBase::AddExpectedMessage
#jira UE-190032
#rb Chris.Constantinescu, Rob.Huyett

[CL 27118558 by Jerome Delattre in ue5-main branch]
2023-08-15 16:21:05 -04:00
christopher waters
e457524ce3 Addressing casing issues with Automation test results by using the casing from the map's path instead of the map name itself since that is more likely to be a different case on clients.
#jira UE-187814
#rb jeannoe.morissette

[CL 26574385 by christopher waters in ue5-main branch]
2023-07-25 11:08:02 -04:00
guillaume abadie
80b3fd8eaa Makes default r.ScreenPercentage settings configurable in project settings per renderer
Motivations of this change:
1) Temporal upscale by default in game builds even when dynamic resolution is not supported;
2) Fixes "Editor Preferences > Performance > Viewport Resolution > Override game screen percentage settings with editor settings in PIE" or (r.Editor.Viewport.OverridePIEScreenPercentage) that used to override r.ScreenPercentage in PIE by default to prevent accidental out of video memory when going PIE on high resolution/DPI display that was not discoverable enought user and generating misunderstanding;
3) Makes editor UI more understandable that the resolution controls under Settings > Engine Scalability Settings is actually PIE resolution settings, since the editor viewport screen percentage is controled independently in editor viewport top left menus;
4) Fixes inconsistency where VR and mobile renderer viewport didn't have their own settings like path tracer or non-realtime editor viewports had;
5) Adds resolution quality presets: just a slider for the 3D rendering resolution is very subjective to a player what to choose, whereas presets allows to more intuitively suggests trade off on the screen percentage range.

What this changes does to the end user:
1) Implements default screen percentage project settings in Project Preferences > Rendering > Default Screen Percentage (or r.ScreenPercentage.Default.*) for each indivual uses case that are: desktop renderer, mobile renderer, VR rendering, path tracer
1.1) Desktop renderer renderer default to screen percentage based on display resolution to automatically temporally upscale;
1.2) All other rendering methods (mobile render, VR renderer, path tracer) defaults to native resolution rendering as before.
2) Adds editor viewport screen percentage settings for mobile renderer and VR in Project Settings > Editor Performance > Viewport Resolution
3) Changes defaults of sg.ResolutionQuality and r.ScreenPercentage to 0 and fallbacks to project settings' default screen percentage
4) Rewrite Level Editor UI > Settings > Engine Scalability Settings to:
4.1) Says it only apply to PIE viewport, not including Simulate in Editor since these is still viewed with editor viewports, and warn when there is no PIE game viewports;
4.2) Explicitly says what is the current screen percentage of the PIE game viewport;
4.3) Explicitly says what is the current rendering and display resolutions;
4.4) Says what rendering mode is being applied (Desktop renderer, mobile renderer, VR rendering, path tracer)
4.5) Says where the settings come from, r.ScreenPercentage was directly set by console code, or whether the Engine Scalability was overident (sg.ResolutionQuality>0) or where the default settings come from (project's default screen percentage settings or editor viewport preferences)
4.6) Says the actually setting is being used (Manual, Based on Display Resolution, Based on OS DPI Scale)
4.7) Explicitly says which anti-aliasing method is being used on the anti-aliasing scalability group
5) The "Editor Preferences > Performance > Viewport resolution > Override project's default screen percentage settings with editor viewports' settings in PIE" (r.Editor.Viewport.OverridePIEScreenPercentage) now chooses whether the default screen percentage project settings should be overriden by the editor viewport's default settings. sg.ResolutionQuality and r.ScreenPercentage now behaving as a override, it means they are also applied in PIE to avoid user confusions when they do indeed what to take control of it in the project settings or game logic.
6) Adds resolution quality presets in BaseScalability.ini that can be customised in a project's DefaultScalability.ini:
6.1) Default 3D resolution preset: Sets sg.ResolutionQuality=0 to end up using the default project settings presets which for the case of the desktop renderer means temporal upscaling by default based on the display resolution like Fortnite's Recommended settings.
6.2) Performance 3D resolution preset: Sets sg.ResolutionQuality=100*1080/2160
6.3) Balanced 3D resolution preset: Sets sg.ResolutionQuality=100*1260/2160
6.4) Quality 3D resolution preset: Sets sg.ResolutionQuality=100*1440/2160
6.5) Native 3D resolution preset: Sets sg.ResolutionQuality=100

#rb rob.srinivasiah, jack.porter, juan.canada, allan.bentham
#jira UE-184651

[CL 25988162 by guillaume abadie in ue5-main branch]
2023-06-14 14:28:12 -04:00
Nathan Hodges
242da1f8e7 Add function to reset GroundTruthData
#preflight 64396f04c947f6523a5b60ef
#rb jerome.delattre
#rnx
#jira none

[CL 25042824 by Nathan Hodges in ue5-main branch]
2023-04-14 12:43:19 -04:00
Tim Smith
b5ac9c22cb Changed FrameDelay to an int32 which is allowed by blueprints. C# UHT wasn't generating an error.
#rb Chris.Kulla
#preflight 6310f54d31667117dfca0e28

[CL 21744521 by Tim Smith in ue5-main branch]
2022-09-01 14:17:12 -04:00
chris kulla
80f833b1bd Expose the number of frames of delay before taking a screenshot
The code had a hardcoded 5 frame delay. All this patch does is expose a setting to control this value (with the default being 5 for backwards compatibility).

The logic is unchanged, first the time based delay is checked, then the frame based delay is checked. You can set either delay to 0 to only use one of them.

This new feature will be used in path tracing related tests to ensure proper convergence regardless of the speed of the machine running the test and without having to set overly lengthy time delays which slow down faster machines without providing guarantees on slower machines.

I have verified that the feature works as expected on all the path tracing tests, including the ones that were frequently failing due to being on the threshold of running too slowly to accumulate enough samples by the time delay.

#jira UE-159434
#rb Jerome.Delattre
#preflight 630fb989556fc14dce9f35a5

[CL 21726693 by chris kulla in ue5-main branch]
2022-08-31 15:47:04 -04:00
bryan sefcik
de1956f47b Ran IWYU on Public headers under Engine/Source/Developer/...
Headers are updated to contain any missing #includes needed to compile and #includes are sorted.  Nothing is removed.

#ushell-cherrypick of 21064294 by bryan.sefcik
#jira
#preflight 62d5c2111062f2e63015e598

#ROBOMERGE-OWNER: bryan.sefcik
#ROBOMERGE-AUTHOR: bryan.sefcik
#ROBOMERGE-SOURCE: CL 21155249 via CL 21158121 via CL 21161259
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21182053 by bryan sefcik in ue5-main branch]
2022-07-20 12:03:45 -04:00
Robb Surridge
0a5a02c5b2 Coding standard fixes: gender-inclusive language
#jira UE-156429
#preflight 62b32e826a25ba6ae52f94bd
#rb jason.walter

[CL 20795375 by Robb Surridge in ue5-main branch]
2022-06-23 11:14:07 -04:00
dave jones2
1d36b62d36 UE-148463 - Update cumulative time tracking Engine variables to doubles
As a part of the LWC work, this change primarily focues on updating FGameTime and UWorld to use doubles for accumulated time. Note that time deltas will remain as floats.

Additionally:

* A few time-related BP functions were updated to use doubles where appropriate. as well as the FRenderThreadUpdateContext struct for the Slate renderer.
* Some compilation errors were also fixed due to the change.
* Minor changes of float literals to drop the 'f' suffix.

#rb andrew.davidson
#jira UE-148463
#preflight 62603748e30cb43e8cbf2aa7

[CL 19866207 by dave jones2 in ue5-main branch]
2022-04-22 11:57:28 -04:00
guillaume abadie
57bcbe4c7c Disables dynamic res on automated screenshot tests
#rb trivial

#ROBOMERGE-AUTHOR: guillaume.abadie
#ROBOMERGE-SOURCE: CL 19627950 via CL 19629834 via CL 19634036
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19635388 by guillaume abadie in ue5-main branch]
2022-04-05 19:23:53 -04:00
daniel coelho
012341dab7 Fix build breakage due to missing include on cl 18977675.
#jira none
#fyi Anousack.Kitisa
#preflight 620a8802583261b0a65b576c

[CL 18981316 by daniel coelho in ue5-main branch]
2022-02-14 12:18:10 -05:00
daniel coelho
0f0df12d27 USD: Improve reproducibility of automated tests.
Implement new function for switching view mode and visualize buffer before taking screenshots;
Screenshot individual buffers instead of using lighting on screenshot tests;
Fix import tests using whatever render context to import was serialized into the config for the import tests;
Disable geometry cache notification on other geometry cache tests;
Increase tolerance on test_pie_streaming_level to give more time for levels to stream in and out before checking for loaded actors;

#jira UE-138139
#rb Anousack.Kitisa, Jerome.Delattre
#preflight 620a448cd332bab853185b48

[CL 18977675 by daniel coelho in ue5-main branch]
2022-02-14 07:15:49 -05:00
guillaume abadie
b000ccbd8e Changes default screen percentage behavior in editor to take more advantage of TSR's upscaling behavior already enabled by default
1) Editor viewports now display at high DPI by default for better Gen5-console feeling in editor on high DPI monitors.

2) Realtime editor viewports now display at screen percentage based on the display pixel count. Fix screen percentage independent of the pixel range could either render not enough pixel at low display resolution or be too slow at high display resolution. Given the level editor viewport can scale in size based on editor user layout preference, monitor, the screen percentage automatically scalling based on display pixel count gives a better experience of quality consistency/GPU cost for rendering frame, especially when going level editor fullscreen with F11 or not. The curve to compute rendering resolution from display resolution is in BaseEngine.ini

3) Non-realtime editor viewports still display at a screen percentage based on OS DPI scale, independent of the display pixel count because use spatial upscaler and therefore need consistent sharpness not worst than editor fonts, but keep same rendering resolution as before to not increase out of GPU memory errors. The performance of potentially rendering at high resolution than a realtime viewport is ignored given the non real-time viewports are not rendering every frame.

4) PIE viewports now override r.ScreenPercentage by default to have consistent resolution quality and performance between realtime editor viewport and gameplay viewport by default without risk of introducing out of GPU memory errors loosing possibly unsaved editor work if the game's graphic settings default to a screen percentage higher. To advertise discoverability of this change, any update to the r.ScreenPercentage cvar will lead to a message display in console/log if overriden by editor settings or dynamic resolution settings.

5) As much as a project can change the default anti-aliasing method, can also configure on a per project basis the default screen percentage behavior in project preference that the editor user can then override locally for his/her own use with his editor settings. For instance mobile only or forward renderer MSAA only video games.

6) When working very high end work-in progress rendering tech project in a econemic context where lattest GPUs are hard to get, the range of GPU performance can vary greatly between contributor of the same projects. To provide more robustness for lowerend GPU of collegues with tech not yet scaling, the project settings now offer a MaxRenderingResolution for editor viewports that can also be overriden on a per user basis.

7) Automated screenshot already takes care of forcing r.ScreenPercentage.

#rb none
#preflight 61e58d4a873f2ea48f33facd

#ROBOMERGE-AUTHOR: guillaume.abadie
#ROBOMERGE-SOURCE: CL 18633512 in //UE5/Release-5.0/... via CL 18633532 via CL 18633538
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18637384 by guillaume abadie in ue5-main branch]
2022-01-18 04:31:45 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00
aurel cordonnier
dc4bf61540 Merge from Release-Engine-Staging @ 17030559 to Release-Engine-Test
This represents UE4/Main @ 17030256 and Dev-PerfTest @ 17030553

[CL 17031509 by aurel cordonnier in ue5-release-engine-test branch]
2021-08-03 11:56:47 -04:00
guillaume abadie
eadb45ef37 Replaces r.PostProcessAAQuality with r.FXAA.Quality & r.TemporalAA.Quality
Now only r.AntiAliasingMethod prevails to selects the anti-aliasing method explicitly for the deferred shading renderer. Each AA technic have it's own cvars to control their quality independently

FXAA now have its own r.FXAA.Quality. The FXAA shader permutations where mis configured with r.PostProcessAAQuality=1 & 2 both mapping to the FXAA_PC_CONSOLE=1 that is now r.FXAA.Quality=0. Instead r.FXAA.Quality now offer more mid-quality settings for FXAA_PC=1 with r.FXAA.Quality=1 & 2 & 3. Backward comaptible migration is as followed:
- r.AntiAliasingMethod=1 r.PostProcessAAQuality=0 -> r.FXAA.Quality=0
- r.AntiAliasingMethod=1 r.PostProcessAAQuality=1 -> r.FXAA.Quality=0
- r.AntiAliasingMethod=1 r.PostProcessAAQuality=2 -> r.FXAA.Quality=0
- r.AntiAliasingMethod=1 r.PostProcessAAQuality=3 -> r.FXAA.Quality=3 but with some neighborhood search setting changed in FXAAShader.usf
- r.AntiAliasingMethod=1 r.PostProcessAAQuality=4 -> r.FXAA.Quality=4
- r.AntiAliasingMethod=1 r.PostProcessAAQuality=5 -> r.FXAA.Quality=5

TAA now have its own r.TemporalAA.Quality. Backward compatible migration is as followed:
- r.AntiAliasingMethod=2 r.PostProcessAAQuality=0 -> r.AntiAliasingMethod=0
- r.AntiAliasingMethod=2 r.PostProcessAAQuality=1 -> r.AntiAliasingMethod=1 r.FXAA.Quality=0
- r.AntiAliasingMethod=2 r.PostProcessAAQuality=2 -> r.AntiAliasingMethod=1 r.FXAA.Quality=0
- r.AntiAliasingMethod=2 r.PostProcessAAQuality=3 r.TemporalAAUpsampleFiltered=0 -> r.TemporalAA.Quality=0
- r.AntiAliasingMethod=2 r.PostProcessAAQuality=3 r.TemporalAAUpsampleFiltered=1 -> r.TemporalAA.Quality=1
- r.AntiAliasingMethod=2 r.PostProcessAAQuality=4 -> r.TemporalAA.Quality=2

MSAA r.AntiAliasingMethod=3 & TSR r.AntiAliasingMethod=4 remains unchanged.

sg.AntiAliasingQuality now maps to different r.TemporalAA.Quality or r.FXAA.Quality

Automated tests on base CL: https://horde.devtools.epicgames.com/job/60d5b8410123b700014f9db5
Automated tests on change CL: https://horde.devtools.epicgames.com/job/60d47cde57b302000114bebf

#rb none
[FYI] jack.porter, wei.liu
#lockdown michal.valient

#ROBOMERGE-SOURCE: CL 16823623 via CL 16823646
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16823657 by guillaume abadie in ue5-release-engine-test branch]
2021-07-12 07:13:12 -04:00
Jerome Delattre
37e79a5b0e Expose FAutomationTestBase::AddTelemetryData to blueprint and python
#jira UEENGQA-38047
#rb none
#rnx

[CL 15681132 by Jerome Delattre in ue5-main branch]
2021-03-11 17:05:29 -04:00
Jerome Delattre
d97e9e2b15 Automated test - Allow image comparison of arbitrary image data
Implement:
 RequestImageComparison(FString& ImageName, int32 Width, int32 Height, TArray<FColor>& ImageData, EAutomationComparisonToleranceLevel Tolerance, FString& Notes);

#jira UEENGQA-38049
#rb William.Ewen

[CL 15465827 by Jerome Delattre in ue5-main branch]
2021-02-18 19:59:25 -04:00