[FYI] brendan.lienau
Original CL Desc
-----------------------------------------------------------------
[Gauntlet] Make test executor mark lack of devices as an error instead of a warning
[CL 27792232 by brendan lienau in ue5-main branch]
* Implement a way to write and run top-down tests in Gauntlet easily.
* Genericize and write tools around existing RPC functionality to allow it to be easily used.
* Create a generic timer that can enable test execution to wait for specific events and time out if they take too long.
**Note** Tests pass, but roles often shows failed due to the test finishing before the Unreal Process(es) have initialized.
#preflight n/a
#rb
[REVIEW] [at]gary.yuan [at]eric.knapik [at]ben.salem [at]zorin.abner
[CL 23935012 by tim kennedy in ue5-main branch]
+ add more arguments to filter out
#preflight 6373abd81c114bec05ac2458
#rnx
#jira none
#rb Chris.Constantinescu
[CL 23159710 by Jerome Delattre in ue5-main branch]
Have gauntlet display the command a test was ran with
Add server output to the horde log so errors can be easier identified from a webpage.
Fixing a typo with Text which should have been Test
#preflight 63408864587d6afac86da474
#rb: Gary.Yuan
https://p4-swarm.epicgames.net/reviews/22408413/
[CL 22451857 by eric knapik in ue5-main branch]
#rb andrew.grant,j.baumgartner,clayton.langford,tony.louthan
#ROBOMERGE-OWNER: ben.salem
#ROBOMERGE-AUTHOR: ben.salem
#ROBOMERGE-SOURCE: CL 19435399 via CL 19435736 via CL 19437281 via CL 19439090 via CL 19439629
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)
[CL 19442469 by ben salem in ue5-main branch]
* Implement loading Json report and resume from last know failure
* Make Gauntlet uses the retry state when critical failure occured on UE automation test.
* Capture the callstack if any and update the json report
* Attached passed log before retrying
* Use enum in json serialization and desirialization
* Use one common Event class for json report and parsing from the log
* Use an unique path for artifact at each runs/retries
* Use Json report to generate the pass summary in Gauntlet
#jira UEENGQA-38051
#rb Chris.Constantinescu, Eric.Knapick
[CL 17160450 by Jerome Delattre in ue5-main branch]
Introduce Gauntlet.ITestReport interface
- enable attaching artifact to report through ITestReport.AttachArtifact(string)
- enable reporting errors and warnings to ITestReport
Deprecate CreateReport(TestResult, Context, Build, Artifacts, ArtifactPath) and favor to CreateReport(ITestReport)
Generate a simplified report for Horde Test Data
Better support copying artifacts with long path on Windows
#jira none
#rb Andrew.Grant
[CL 14859929 by Jerome Delattre in ue5-main branch]
Avoid redundant error report in the log by implementing a new property LogWarningsAndErrorsAfterSummary to ITestNode
#jira none
#rnx
#rb josh.engebretson, andrew.grant
[CL 14809680 by Jerome Delattre in ue5-main branch]
