Commit Graph

218 Commits

Author SHA1 Message Date
ben salem
8c3eb77842 Move some substantial RPC C# classes from Fortnite to Engine so other teams can start using them.
#rb gary.yuan

[CL 34501043 by ben salem in ue5-main branch]
2024-06-19 11:16:46 -04:00
bob tellez
ed948eb0ec [Backout] - CL34433305
[FYI] Ben.Salem
Original CL Desc
-----------------------------------------------------------------
Move the core Gauntlet RPC code to engine so other projects can use it. Also remove old dead code that was causing namespace collisions!

[CL 34441676 by bob tellez in ue5-main branch]
2024-06-17 17:31:09 -04:00
ben salem
553912947a Move the core Gauntlet RPC code to engine so other projects can use it. Also remove old dead code that was causing namespace collisions!
[CL 34438145 by ben salem in ue5-main branch]
2024-06-17 15:43:27 -04:00
jerome delattre
66d7bd0e7f [Gauntlet] Fix Metadata duplication in legacy Horde Report
#rnx

[CL 34383609 by jerome delattre in ue5-main branch]
2024-06-14 16:41:00 -04:00
jerome delattre
db7c136d6c [Gauntlet] Revert change of case for Metadata property in legacy Horde reports
#rnx

[CL 34370071 by jerome delattre in ue5-main branch]
2024-06-14 10:02:40 -04:00
brendan lienau
5f71d65325 [Gauntlet] Adds IBuildValidtor type to support a user defined way of selecting the "Latest Known Good" (LKG) build
To use this, simply define a type that Implement IBuildValidator and override GetLatestGoodBuild() to return a string to the path of the latest known good build
Then, run a gauntlet test where -build="LKG" OR -build="LatestGood"

#rb Ryan.Bazinet

[CL 34360418 by brendan lienau in ue5-main branch]
2024-06-13 19:55:35 -04:00
jerome delattre
481304d310 [Gauntlet] Horde Test Report Moving version and metadata properties into data blob property
#rnx
#rb rob.huyett

[CL 34336596 by jerome delattre in ue5-main branch]
2024-06-13 08:33:30 -04:00
brendan lienau
884d374fe2 [Gauntlet] Fix CIS with params function
[CL 34281248 by brendan lienau in ue5-main branch]
2024-06-11 12:43:55 -04:00
brendan lienau
87d8054c6f [Gauntlet] Move FullClean and SkipInstall static properties to DevicePool and out of UnrealAppConfig
This mainly improves separation of concern. Additionally, desktops should now be able to skipinstall on build machines not using reservation blocks.

#rb marcelo.gomes

[CL 34281091 by brendan lienau in ue5-main branch]
2024-06-11 12:38:01 -04:00
joe kirchoff
72a8c5996d Fix various XML comment issues in AutomationTool and various scripts (non-functional change)
#rnx

[CL 34263204 by joe kirchoff in ue5-main branch]
2024-06-10 21:51:45 -04:00
jerome delattre
0d081b79f8 Introduce TestData v2 output
- Move previous iterations of the HordeReport to a legacy implementation section
- Reimplement AutomatedTestSessionData base TestData v2 data model
- Change ITestReport AddEvent signature in favor of using formating with params keyword
- Remove redundancy of ITestEvent based property (UnrealTestNode.Events)
- Use Dictionary for Artifact Files in UnrealAutomatedTestResult json data model instead of specialized image comparison type

#jira UE-214378
#rnx
#rb adam.miezianko, sean.sweeney, sebastian.lewicki

[CL 34234717 by jerome delattre in ue5-main branch]
2024-06-10 09:18:33 -04:00
jerome delattre
75b7496ef4 Prune log info when passing test results to test report when test succeed to reduce foot print
The optimization is enabled by default but can be disabled through project config:
[/Script/AutomationController.AutomationControllerSettings]
bPruneLogsOnSuccess=false

Sorting tests by failure type in json output is now disabled by default and can be enabled using this config:
bSortTestsByFailure=true
It resides in the same AutomationControllerSettings section.

#jira UE-215907
#rb sebastian.lewicki

[CL 34092189 by jerome delattre in ue5-main branch]
2024-06-04 09:23:22 -04:00
joe kirchoff
eba691c1ec UAT scripts: Assembly.SafeGetLoadedTypes()
#rnx

[CL 34051682 by joe kirchoff in ue5-main branch]
2024-05-31 20:47:53 -04:00
brendan lienau
006a943f56 [Gauntlet] When running a single UnrealTestNode, the Horde Summary will now include the full artifact path to the specific test
#rb Jerome.Delattre

[CL 33954629 by brendan lienau in ue5-main branch]
2024-05-28 17:39:55 -04:00
brendan lienau
4a6d575091 [Gauntlet] Cleanup/Re-order of TargetDeviceiOS properties/functions
#jira UE-201389

[CL 33843129 by brendan lienau in ue5-main branch]
2024-05-22 16:33:07 -04:00
brendan lienau
debd523fdb [Gauntlet] Fix -dev executables not working on desktops when running without -ExperimentalLaunchFlow
[CL 33731167 by brendan lienau in ue5-main branch]
2024-05-17 14:22:09 -04:00
jerome delattre
f559c48c57 [Gauntlet] Add DateTime to ITestEvent
#rnx
#jira UE-210260

#rb sebastian.lewicki

[CL 33550354 by jerome delattre in ue5-main branch]
2024-05-09 13:03:15 -04:00
brendan lienau
0b7acffefd [Gauntlet] Adds -dev support for Desktop when running with ExperimentalLaunchFlow
[CL 33351564 by brendan lienau in ue5-main branch]
2024-04-30 16:26:14 -04:00
brendan lienau
38dbb40f53 [Gauntlet] Adds -dev local binarry overrides (Overlay Executables) when running with -ExperimentalLaunchFlow
- Adds the OverlayExecutable type
    - Adds OverlayExecutable member to UnrealAppConfig. Only initialized if -dev is present
    - This object can check if there is a local newer binary for the relevant Role and return the path to it


#rb adolf.stary, clayton.langford

[CL 33350955 by brendan lienau in ue5-main branch]
2024-04-30 16:13:24 -04:00
brendan lienau
5ddd1409b8 [Gauntlet] Direct windows crashdump generation on process kill to saved crashes directory
[CL 33350668 by brendan lienau in ue5-main branch]
2024-04-30 16:09:32 -04:00
brendan lienau
0ce265bf7e [Gauntlet] Fix CI errors from crash dump removal merge
[CL 33348557 by brendan lienau in ue5-main branch]
2024-04-30 15:26:57 -04:00
brendan lienau
297a7c8570 [Gauntlet] Remove legacy method of crash dump copying during SaveRoleArtfacts
Each platform already handles crash dump copying during the ArtifactPath accessor

[CL 33347850 by brendan lienau in ue5-main branch]
2024-04-30 15:07:16 -04:00
jerome delattre
6bf5cf1052 [Gauntlet] (On Windows) Generate mini dump when killing application if timeout is detected.
#rnx
#jira UE-212994
#rb brendan.lienau, sebastian.lewicki

[CL 33232014 by jerome delattre in ue5-main branch]
2024-04-25 11:05:26 -04:00
olivier lamothe
f608fc05b3 Gauntlet: Fix LogFile sometimes not being set in DesktopCommonAppInstall
When the RegEx matches the first group, the LogFile name was empty

#rb ryan.bazinet

[CL 33229422 by olivier lamothe in ue5-main branch]
2024-04-25 09:29:10 -04:00
jon olick
da218d2bca Instead of passing in AppConfig to Run, store AppConfig in AndroidInstall.
#rb Brendan.Lienau, Chris.Babcock

[CL 33050622 by jon olick in ue5-main branch]
2024-04-17 17:45:03 -04:00