Called when entering and leaving test section.
ie:
For a test named MyTest.Log.Validation.TestTrue
These will be called through OnEnteringTestSectionEvent before any test of the corresponding section is executed:
- MyTest
- MyTest.Log
- MyTest.Log.Validation
And the corresponding one when leaving the section after execution.
One can register a callback this way:
FAutomationTestFramework::Get().GetOnEnteringTestSection(TEXT("MyTest.Log")).AddRaw(this, &FMyModule::MySectionCallback);
As first argument the name of the section is passed.
#rb Sebastian.Lewicki
[CL 29323495 by jerome delattre in ue5-main branch]
+ Improve messaging when movie pipeline test is failing for reason other than image comparison.
+ Store ScreenshotName inside comparison request and result.
#jira UE-194701
#rnx
#rb Matt.Hoffman, Sean.Sweeney
[CL 28750594 by jerome delattre in ue5-main branch]
#fyi Jerome.Delattre
#jira FORT-617474
Original CL Desc
-----------------------------------------------------------------
Add Support for shader model in automation test framework exclusion list
#jira UE-185221
#preflight 646685c83660522200415d43
#rnx
#rb Chris.Constantinescu
[CL 25715585 by keaton stewart in ue5-main branch]
Adds the value of FApp::GetInstanceId() to the instance field in index.json such that it is now (hostname)-(session ID)-(instance ID)
#jira UE-174430
#preflight 63caca55a265157a6d4fa37b
#rb Jerome.Delattre
[CL 23835184 by hach-que in ue5-main branch]
These were using Matinee before, which has been removed. These automation classes would have to be rewritten using sequencer, but level sequence actors are in a different module.
#rb andrew.rodham, matt.hoffman
#jira UE-105313
#preflight 63586c21555771b1419af8ff
[CL 22803522 by ludovic chabant in ue5-main branch]
This change completely removes Matinee from the codebase. The data types and runtime evaluation code are gone. The "InterpEdit" editor mode, asset factories, and other editor support for Matinee is gone. CameraAnims and their sequencer tracks are gone. FBX import/export for Matinee assets is gone.
#preflight 62d0591e562520a394bca1d1
#preflight 62d194f63c3df323904679c1
#jira UE-105313
#rb none
#ROBOMERGE-OWNER: max.chen
#ROBOMERGE-AUTHOR: ludovic.chabant
#ROBOMERGE-SOURCE: CL 21116312 via CL 21116339 via CL 21118051
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)
[CL 21142481 by max chen in ue5-main branch]
+ Add support for device tagging through Automation Worker
#preflight 62b1fd0dda272a658f573d57
#rb Chris.Constantinescu
#jira none
[CL 20859264 by Jerome Delattre in ue5-main branch]
* Rename blacklist to excludelist (requested by high management)
* Support section exclusion rule to be able to exclude entire section of tests (ie PathTracing is only supported on Win64 for now)
* Mark excluded test as skipped in the report instead of entirely removed for test list. Check for exclusion just before running the test.
* Remove NotEnoughParticipant state in favor of Skipped (same conditions lead to Skipped state with appropriate messaging)
* Add support for exclusion management from the Test Automation window. (added a column at the end of each row)
* Expose device information to UE test report
* Add support for metadata in Gauntlet test report for Horde
Limitations:
* Management through the UI is limited to which test is available through in the active worker node. That's mean Runtime only tests are not listed from a worker that is Editor(the default) and platform specific are not clearly identified.
* For platforms, the mechanic to access their config and save it will remain to be done. In the meantime, it needs to be done manually through the target platform config file.
#jira UE-125960
#jira UE-125974
#rb Chris.Constantinescu, Eric.Knapik, Louise.Rasmussen
[CL 17607554 by Jerome Delattre in ue5-main branch]
* Isolate code for compressing in PNG and use this code for screenshot image compression
* Mark screenshot as not similar when error message is not empty when considering comparison result
#rnx
#jira UE-121446
#rb Julien.StJean, Chris.Contantinescu
[CL 17138306 by Jerome Delattre in ue5-main branch]
