42 Commits

Author SHA1 Message Date
sergei svetlov
c739c0cb35 Clean up screenshot comparison report pipeline
#rb Jerome.Delattre
#jira UE-200919
#rnx

[CL 30687983 by sergei svetlov in ue5-main branch]
2024-01-18 12:22:38 -05:00
christopher fiala
d5ec52b3cc Add a new CVar, Automation.LightweightStereoTestVariants, which stops variants from running if we've already encounted an error, and skips saving/attaching screenshots for successful variants to save space. Enabled by default. Also moves the "Supports Stereo Test Variants" checkbox to the "Screenshot" section.
#jira UE-194361
#rb jerome.delattre, chris.constantinescu

[CL 29718813 by christopher fiala in ue5-main branch]
2023-11-14 13:31:30 -05:00
oleg krasnov
2eb50ce77a SessionFrontend - Make FScreenShotManager::OpenComparisonReports and metadata loading async
#jira UE-195234
#rb Jerome.Delattre

[CL 28908278 by oleg krasnov in ue5-main branch]
2023-10-19 02:01:14 -04:00
jerome delattre
4a5957a406 Add frame comparison failure to the test report when comparing image sequence for Movie Pipeline Functional Test
+ Improve messaging when movie pipeline test is failing for reason other than image comparison.
+ Store ScreenshotName inside comparison request and result.

#jira UE-194701
#rnx
#rb Matt.Hoffman, Sean.Sweeney

[CL 28750594 by jerome delattre in ue5-main branch]
2023-10-13 09:42:36 -04:00
Jerome Delattre
06389d5027 Support Confidential platforms in Screenshot fallback settings
#jira UE-163243
#preflight 646e5f421417daba1e01a8eb
#rnx
#rb Brian.White

[CL 25643793 by Jerome Delattre in ue5-main branch]
2023-05-26 11:41:50 -04:00
jerome delattre
80b32adb45 Prevent approved reference images to be copied to html test report for successful test
#jira UE-176364
#preflight 63e3d66d8374ef68dffe3565
#rb Chris.Constantinescu

[CL 24085750 by jerome delattre in ue5-main branch]
2023-02-08 18:20:35 -05:00
aidin abedi
88e9765627 Improve ImageComparer by extending FComparableImage to be more straightforward when comparing images that are only in-memory.
#jira UE-173166
#preflight 63c502d70225f00e14f28aff
#review @yuriy.odonnell

[CL 23776626 by aidin abedi in ue5-main branch]
2023-01-19 12:51:53 -05:00
Zak Middleton
84fb0737e0 #ue5 - Fix type conversion warnings in modules: AutomationController, AutomationTest, AVEncoder, AVIWriter.
#jira UE-160830
#rb Andrew.Davidson
#preflight 630d2147556fc14dce527da5

[CL 21690862 by Zak Middleton in ue5-main branch]
2022-08-29 18:32:03 -04:00
bryan sefcik
de1956f47b Ran IWYU on Public headers under Engine/Source/Developer/...
Headers are updated to contain any missing #includes needed to compile and #includes are sorted.  Nothing is removed.

#ushell-cherrypick of 21064294 by bryan.sefcik
#jira
#preflight 62d5c2111062f2e63015e598

#ROBOMERGE-OWNER: bryan.sefcik
#ROBOMERGE-AUTHOR: bryan.sefcik
#ROBOMERGE-SOURCE: CL 21155249 via CL 21158121 via CL 21161259
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21182053 by bryan sefcik in ue5-main branch]
2022-07-20 12:03:45 -04:00
Matt Hoffman
46e568c199 Automation: Exposed "GetIdealApprovedFolderForImage" to IScreenShotManager so that external modules can reuse the same code to correctly calculate where to store a ground truth image.
#jira UE-105318
#rb Jerome.Delattre
#preflight 61f2ee611f12674afbbe9f6d

[CL 18762242 by Matt Hoffman in ue5-main branch]
2022-01-27 16:30:09 -05:00
jerome delattre
6490e355e2 Properly initiate FImageComparisonResult::CreationTime member
Fix originaly proposed through PR #8632. But the fix is not to exclude the class member to be tested but to properly intiate it.

#jira UE-135340
#rb Julien.StJean

#ROBOMERGE-AUTHOR: jerome.delattre
#ROBOMERGE-SOURCE: CL 18286341 in //UE5/Release-5.0/... via CL 18286368
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18286382 by jerome delattre in ue5-release-engine-test branch]
2021-11-24 13:36:04 -05:00
yuriy odonnell
9b78219a4f Remove FComparableImage::Process() and its outputs (Total/Average values).
The outputs were computed incorrectly (due to data races in ParallelFor body) and inefficiently (looping over pixel columns). Luckily, outputs were not referenced by any subsequent code.

Saves ~0.3 seconds of processing time (~30%, ~1.0s sec -> ~0.7s) per screenshot test on AMD TR 3970X.

#rb Arciel.Rekman

#ROBOMERGE-AUTHOR: yuriy.odonnell
#ROBOMERGE-SOURCE: CL 18195320 in //UE5/Release-5.0/... via CL 18195381
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 18195392 by yuriy odonnell in ue5-release-engine-test branch]
2021-11-15 15:00:48 -05:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
jerome delattre
5ce6ba8080 Test Autoamtion - fix incoming screenshot image saved as JPEG instead of PNG since jpg format was enabled for thumbnail
* Isolate code for compressing in PNG and use this code for screenshot image compression
* Mark screenshot as not similar when error message is not empty when considering comparison result

#rnx
#jira UE-121446
#rb Julien.StJean, Chris.Contantinescu

#ROBOMERGE-SOURCE: CL 17138306 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17138314 by jerome delattre in ue5-release-engine-test branch]
2021-08-11 13:39:11 -04:00
aurel cordonnier
02c0f425e8 Copy up from Release-Engine-Staging @ 16738359
This represents UE4/Main @ 16738161 and Dev-PerfTest @ 16737719

[CL 16738582 by aurel cordonnier in ue5-release-engine-test branch]
2021-06-22 00:27:54 -04:00
Marc Audy
e80ea6b959 Merge from Release-Engine-Staging @ 16444985
This represents UE4/Main @ 16445039 and Dev-PerfTest @ 16444526

[CL 16445122 by Marc Audy in ue5-release-engine-test branch]
2021-05-25 02:43:26 -04:00
Marc Audy
11f5b21210 Merging //UE5/Release-Engine-Staging @ 13752110 to Main (//UE5/Main)
#rnx

[CL 13753156 by Marc Audy in ue5-main branch]
2020-06-23 18:40:00 -04:00
Rolando Caloca
bbb9564388 Copying //UE4/Dev-RenderPlat-Staging@11110326 to //UE4/Main
#rb none
#rnx

[CL 11110369 by Rolando Caloca in Main branch]
2020-01-24 18:07:01 -05:00
ryan durand
471d972e62 Updating copyright for Engine Developer.
#rnx
#rb none


#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)

[CL 10870584 by ryan durand in Main branch]
2019-12-26 15:32:37 -05:00
Chris Gagnon
2e87118a18 Copying //UE4/Dev-Editor to Dev-Main (//UE4/Dev-Main) Interim 4.24.
#rb none

[CL 8614014 by Chris Gagnon in Main branch]
2019-09-10 11:35:20 -04:00
zzz77
97e4a99783 PR #6033: Fixes for IntelliSense (Contributed by zzz77)
#rb none
#rnx

[CL 7308593 by Ben Marsh in Dev-Build branch]
2019-07-15 10:09:44 -04:00
Chris Gagnon
8fc25ea18e Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4676797 by Chris Gagnon in Dev-Editor branch]
2019-01-02 14:54:39 -05:00
Ben Marsh
530369c613 Merging //UE4/Dev-Main to Dev-Build (//UE4/Dev-Build)
#rb none
#rnx

[CL 4662695 by Ben Marsh in Dev-Build branch]
2018-12-14 14:49:12 -05:00
Ben Marsh
13d012685f Merging copyright update from 4.19 branch.
#rb none
#rnx
#jira

[CL 3818977 by Ben Marsh in Staging-4.19 branch]
2018-01-02 15:30:26 -05:00
Chris Bunner
d6e57b4ca3 Copying //UE4/Dev-Automation to //UE4/Dev-Main (Source: //UE4/Dev-Automation @ 3792497)
#rb
#lockdown Nick.Penwarden

============================
  MAJOR FEATURES & CHANGES
============================

Change 3776794 by Chris.Bunner

	Fixed inverted check.

Change 3778396 by Chris.Bunner

	By default functional screenshot tests now enable camera cut and a fixed tonemap/exposure.
	Existing tests in PostProcessing have had the FixedTonemapping flag disabled.
	Updated all existing screenshots against the new fixed gamma ground truth.

Change 3778592 by Chris.Bunner

	Updating PostProcess screenshots that managed to collide mid P4 add.

Change 3778793 by Chris.Bunner

	Override the secondary screen percentage (used for high DPI) when taking screenshots. This is necessary for consistent results between machines.

Change 3781715 by Chris.Bunner

	Updating Windows screenshots.

Change 3781717 by Chris.Bunner

	Fixes for eye adaptation and tonemapping override consistency between test types and run modes.

Change 3783199 by Chris.Bunner

	Added separate add/replace screenshot buttons to know if we're working with a matching platform tier or a fallback.

Change 3783228 by Chris.Bunner

	When incoming screenshots are different sizes create a delta of the minimum shared dimensions but still force a failure. The UI is more consistent showing any delta, even if it's almost pure white.

Change 3783712 by Chris.Bunner

	Rebuilt translucency lighting test map and updated screenshot results.

Change 3784010 by Chris.Bunner

	Adding Mac-specific PostProcessing screenshots.

Change 3787456 by Chris.Bunner

	Improving name matching consistency of test blacklist entries.

Change 3787522 by Chris.Bunner

	Updating Mac-specific screenshots.

Change 3787583 by Chris.Bunner

	Updating Mac-Specific screenshots.

Change 3787832 by Chris.Bunner

	Fixing-up NaNs in two saved level's HLOD world settings.

Change 3789147 by Chris.Bunner

	Updating Sequencer sub-levels which still had NaNs in World Settings.

Change 3791454 by Chris.Bunner

	Deleted ancient screenshot re-introduced by Fortnite merge.

Change 3781713 by Chris.Bunner

	Updating platform unique screenshots - Tessellation, fixed vs variable screenshot size, Niagara simulation.

Change 3776756 by Chris.Bunner

	Initial pass at hierarchical screenshot testing.
	Deleted existing platforms automated test screenshots.

Change 3784051 by Chris.Bunner

	Updated CableActor screenshots for new capture defaults.

Change 3787092 by Chris.Bunner

	Added blacklist as "AutomationTestBlacklist" in Engine.ini.
	Formatting fix-up in existing automation test config.
	Removed workaround cvar for physicalized animation tests failing on platforms.

[CL 3792526 by Chris Bunner in Main branch]
2017-12-06 14:51:13 -05:00