* Introduce Automated Test Session data model for Horde TestData
* Introduce Suite and Type to UnrealTestNode to have better control over Session naming
* Add PreflightChange option to discriminate Session that are not official run
* Add test Duration and DateTime to UE json report model
* Accomodate for posting New and Old report models
#jira UEENGQA-29178
#preflight 622bb8ec46679d56c323f49a
#rb Chris.Constantinescu
[CL 19367745 by Jerome Delattre in ue5-main branch]
Added LogAudioMixer for FN as a log that doesn't lead to functional tests having warnings
[REVIEW] [at]eric.knapik
#preflight https://horde.devtools.epicgames.com/job/62085c12c52105b487b8f45a
#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 18974282 via CL 18974333 via CL 18974336 via CL 18974337 via CL 18992583 via CL 18992954
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)
[CL 18997205 by andrew grant in ue5-main branch]
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971
[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
* Rename blacklist to excludelist (requested by high management)
* Support section exclusion rule to be able to exclude entire section of tests (ie PathTracing is only supported on Win64 for now)
* Mark excluded test as skipped in the report instead of entirely removed for test list. Check for exclusion just before running the test.
* Remove NotEnoughParticipant state in favor of Skipped (same conditions lead to Skipped state with appropriate messaging)
* Add support for exclusion management from the Test Automation window. (added a column at the end of each row)
* Expose device information to UE test report
* Add support for metadata in Gauntlet test report for Horde
Limitations:
* Management through the UI is limited to which test is available through in the active worker node. That's mean Runtime only tests are not listed from a worker that is Editor(the default) and platform specific are not clearly identified.
* For platforms, the mechanic to access their config and save it will remain to be done. In the meantime, it needs to be done manually through the target platform config file.
#jira UE-125960
#jira UE-125974
#ROBOMERGE-AUTHOR: jerome.delattre
#ROBOMERGE-SOURCE: CL 17607554 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)
[CL 17607557 by jerome delattre in ue5-release-engine-test branch]
Introduce bElevateLogWarningsToErrors as a good name for existing functionality. Note it's assuming the name as it's known by its customer code. This change lines up the settings variable to its colloquial name around the codebase.
Deprecate bTreatLogWarningsAsTestErrors as it was misleading because it doesn't really shuffle log warnings into the test errors bucket as much as it just reclassifies the log verbosity from log warning to log error.
Remove bTreatLogErrorsAsTestErrors as it wasn't functional outside of EditorIterationTest's custom base class, and it's tempting to use incorrectly in place of bSuppressLogErrors
#rnx
#jira none
#rb andrew.grant
#preflight 6142442c4778fa000135b987
#ROBOMERGE-AUTHOR: geoff.evans
#ROBOMERGE-SOURCE: CL 17526549 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)
[CL 17526590 by geoff evans in ue5-release-engine-test branch]
* Implement loading Json report and resume from last know failure
* Make Gauntlet uses the retry state when critical failure occured on UE automation test.
* Capture the callstack if any and update the json report
* Attached passed log before retrying
* Use enum in json serialization and desirialization
* Use one common Event class for json report and parsing from the log
* Use an unique path for artifact at each runs/retries
* Use Json report to generate the pass summary in Gauntlet
#jira UEENGQA-38051
#rb Chris.Constantinescu, Eric.Knapick
#ROBOMERGE-SOURCE: CL 17160450 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)
[CL 17160467 by jerome delattre in ue5-release-engine-test branch]
The messaging system destroys these objects with FMemory::Free, which has different alignment logic than operator new when ASAN is used, resulting in a crash.
#jira none
#rnx
#rb Jerome.Delattre, Matt.Peters
#ROBOMERGE-SOURCE: CL 17116813 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)
[CL 17116851 by mihnea balta in ue5-release-engine-test branch]
