* Support multiple devices on a test (ie: network tests)
* Support log attached to device app instance
* Make Incomplete run flagged as failure when session is finishing
* Critical failure tag as crash in the test report
#preflight 627bffe71e749933439e54ba
#rb Chris.Constantinescu
#jira none
[CL 20378481 by Jerome Delattre in ue5-main branch]
[REVIEW] [at]eric.knapik
#preflight 624f5058f10bcc0f4f84a6ba
#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 19680155 via CL 19681068 via CL 19681084 via CL 19681091
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)
[CL 19681818 by andrew grant in ue5-main branch]
Not all were converted because they were designed for packaged applications whereas low level tests are designed to run as "native" non-packaged applications.
- reporting support for non-desktop platforms, for the moment Catch2 report type "console" is used that is sent to a .out file
- most number of tests possible running multi platform, slower tests excluded on incremental builds
- slow tests are moved to run on the monolithic build
- Catch2 report failure event listener such that Horde detects them as build errors. Must add new Horde matcher for Catch2 failures
#rb Mark.Lintott
#preflight 623c8de389625f0612dabd64
[CL 19498448 by chris constantinescu in ue5-main branch]
#rb andrew.grant,j.baumgartner,clayton.langford,tony.louthan
#ROBOMERGE-OWNER: ben.salem
#ROBOMERGE-AUTHOR: ben.salem
#ROBOMERGE-SOURCE: CL 19435399 via CL 19435736 via CL 19437281 via CL 19439090 via CL 19439629
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)
[CL 19442469 by ben salem in ue5-main branch]
* Introduce Automated Test Session data model for Horde TestData
* Introduce Suite and Type to UnrealTestNode to have better control over Session naming
* Add PreflightChange option to discriminate Session that are not official run
* Add test Duration and DateTime to UE json report model
* Accomodate for posting New and Old report models
#jira UEENGQA-29178
#preflight 622bb8ec46679d56c323f49a
#rb Chris.Constantinescu
[CL 19367745 by Jerome Delattre in ue5-main branch]
Made DOD self-abort if it thinks there's an issue instead of waiting for the timeout
#preflight https://horde.devtools.epicgames.com/log/622a980f32749f363c5f52d5
[REVIEW] [at]erik.knapik
#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 19347702 via CL 19348799 via CL 19352331 via CL 19353055 via CL 19355827
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)
[CL 19357776 by andrew grant in ue5-main branch]
Fix some test/asserts when running offscreen rendering. We use a GenericWindow which was hitting some fatal logs
#jira none
#rb none
#preflight 620ed89bc91bcd0017af2295
[CL 19044653 by Brandon Schaefer in ue5-main branch]
- skipClient option allows for client steps to be run in UAT with a null role which causes them to be skipped
- the test still runs and is considered successful and all dependencies are still executed
- this results in a "Null" client device with a null platform
- IDeviceUsageReporter causes exceptions with this configuration
- change IDeviceUsageReporter methods to take a platform reference and handle a null platform internally
- no events are logged for the null device and platform
- the null client device associated with teh skip tests will create a NullAppInstance
- NullAppInstance only completes upon WaitForExit() but many tests don't do this as they run until they see a specific line in the output log or something similar (ex: boot tests)
- this causes some NullAppInstance tests to never complete and to run until a timeout is encountered
- NullAppInstance is now considered completed and HasExited upon creation to resolve this
- the base test node then identifies the null app instance as completed
#rb Chris.Constantinescu, Josh.Engebretson, Jerome.Delattre
#jira none
#preflight https://horde.devtools.epicgames.com/job/61e1c430b14a67d611987c49
[CL 18648971 by eric mcdaniel in ue5-main branch]
#jira: none
[at]Andrew.Grant [at]Jerome.Delattre [at]Chris.Constantinescu
#preflight 61d4c51d34785a82bb7569ed
#ROBOMERGE-AUTHOR: eric.knapik
#ROBOMERGE-SOURCE: CL 18520341 via CL 18521813 via CL 18522249 via CL 18522376 via CL 18523520 via CL 18523555 via CL 18523598
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v899-18417669)
[CL 18523633 by eric knapik in ue5-main branch]
