Commit Graph

85 Commits

Author SHA1 Message Date
jerome delattre
19354f4b1a [LLT] Avoid using StdOut directly, use WriteOutputToFile to copy the log and instanciate the Log buffer reader at the start of the test
#rnx

[CL 36389004 by jerome delattre in 5.5 branch]
2024-09-18 09:51:27 -04:00
jerome delattre
2afa890d78 [Gauntlet] Fix false positive when Editor canceled initialization
#rnx

[CL 36286398 by jerome delattre in 5.5 branch]
2024-09-13 18:35:31 -04:00
jerome delattre
aa35ffe37a [Gauntlet] fix partial log file when running client locally
#rnx

[CL 36173574 by jerome delattre in 5.5 branch]
2024-09-10 21:45:12 -04:00
Ryan Hummer
4af2fd066d Updating Dev-Release-5.5 from Main at CL #36144969
#okforversepublic

[CL 36146571 by Ryan Hummer in Dev-5.5 branch]
2024-09-10 10:26:02 -04:00
brendan lienau
751d9dcfe5 [Gauntlet] Updated implementation of TargetDeviceIOS
This CL has iOS reach parity all other TargetDevice platforms in the standard set forth in UE-201403
This includes launch flow function implementations, overlay executable support, and updated toolset to support modern versions of iOS and xCode
Namely, ios-deploy has become obsolete for iOS 17+ due to changes in Apple's system frameworks. Instead Apple's new tool, devicectl, is used in versions of xcode where available

The following types of builds are supported:
- Standard Apps (non-bulk)
- Standard IPAs (non-bulk)
- Bulk apps
- dev apps (bulk and non-bulk) aka overlay executables

#jira UE-201389

#rb chris.constantinescu
#tests Preflight and local runs against all build variations

[CL 35666507 by brendan lienau in ue5-main branch]
2024-08-20 16:03:22 -04:00
jerome delattre
dc3c2fad2d [Gauntlet] point Demos directory to user directory instead of build directory, since Replay feature search in user directory for assets.
#rnx
#rb rob.huyett, sebastian.lewicki

[FYI] brendan.lienau

[CL 35212576 by jerome delattre in ue5-main branch]
2024-07-31 10:15:19 -04:00
jerome delattre
66d7bd0e7f [Gauntlet] Fix Metadata duplication in legacy Horde Report
#rnx

[CL 34383609 by jerome delattre in ue5-main branch]
2024-06-14 16:41:00 -04:00
jerome delattre
db7c136d6c [Gauntlet] Revert change of case for Metadata property in legacy Horde reports
#rnx

[CL 34370071 by jerome delattre in ue5-main branch]
2024-06-14 10:02:40 -04:00
brendan lienau
5f71d65325 [Gauntlet] Adds IBuildValidtor type to support a user defined way of selecting the "Latest Known Good" (LKG) build
To use this, simply define a type that Implement IBuildValidator and override GetLatestGoodBuild() to return a string to the path of the latest known good build
Then, run a gauntlet test where -build="LKG" OR -build="LatestGood"

#rb Ryan.Bazinet

[CL 34360418 by brendan lienau in ue5-main branch]
2024-06-13 19:55:35 -04:00
jerome delattre
481304d310 [Gauntlet] Horde Test Report Moving version and metadata properties into data blob property
#rnx
#rb rob.huyett

[CL 34336596 by jerome delattre in ue5-main branch]
2024-06-13 08:33:30 -04:00
joe kirchoff
72a8c5996d Fix various XML comment issues in AutomationTool and various scripts (non-functional change)
#rnx

[CL 34263204 by joe kirchoff in ue5-main branch]
2024-06-10 21:51:45 -04:00
jerome delattre
0d081b79f8 Introduce TestData v2 output
- Move previous iterations of the HordeReport to a legacy implementation section
- Reimplement AutomatedTestSessionData base TestData v2 data model
- Change ITestReport AddEvent signature in favor of using formating with params keyword
- Remove redundancy of ITestEvent based property (UnrealTestNode.Events)
- Use Dictionary for Artifact Files in UnrealAutomatedTestResult json data model instead of specialized image comparison type

#jira UE-214378
#rnx
#rb adam.miezianko, sean.sweeney, sebastian.lewicki

[CL 34234717 by jerome delattre in ue5-main branch]
2024-06-10 09:18:33 -04:00
brendan lienau
debd523fdb [Gauntlet] Fix -dev executables not working on desktops when running without -ExperimentalLaunchFlow
[CL 33731167 by brendan lienau in ue5-main branch]
2024-05-17 14:22:09 -04:00
jerome delattre
f559c48c57 [Gauntlet] Add DateTime to ITestEvent
#rnx
#jira UE-210260

#rb sebastian.lewicki

[CL 33550354 by jerome delattre in ue5-main branch]
2024-05-09 13:03:15 -04:00
brendan lienau
0b7acffefd [Gauntlet] Adds -dev support for Desktop when running with ExperimentalLaunchFlow
[CL 33351564 by brendan lienau in ue5-main branch]
2024-04-30 16:26:14 -04:00
brendan lienau
38dbb40f53 [Gauntlet] Adds -dev local binarry overrides (Overlay Executables) when running with -ExperimentalLaunchFlow
- Adds the OverlayExecutable type
    - Adds OverlayExecutable member to UnrealAppConfig. Only initialized if -dev is present
    - This object can check if there is a local newer binary for the relevant Role and return the path to it


#rb adolf.stary, clayton.langford

[CL 33350955 by brendan lienau in ue5-main branch]
2024-04-30 16:13:24 -04:00
brendan lienau
5ddd1409b8 [Gauntlet] Direct windows crashdump generation on process kill to saved crashes directory
[CL 33350668 by brendan lienau in ue5-main branch]
2024-04-30 16:09:32 -04:00
brendan lienau
0ce265bf7e [Gauntlet] Fix CI errors from crash dump removal merge
[CL 33348557 by brendan lienau in ue5-main branch]
2024-04-30 15:26:57 -04:00
brendan lienau
297a7c8570 [Gauntlet] Remove legacy method of crash dump copying during SaveRoleArtfacts
Each platform already handles crash dump copying during the ArtifactPath accessor

[CL 33347850 by brendan lienau in ue5-main branch]
2024-04-30 15:07:16 -04:00
jerome delattre
6bf5cf1052 [Gauntlet] (On Windows) Generate mini dump when killing application if timeout is detected.
#rnx
#jira UE-212994
#rb brendan.lienau, sebastian.lewicki

[CL 33232014 by jerome delattre in ue5-main branch]
2024-04-25 11:05:26 -04:00
olivier lamothe
f608fc05b3 Gauntlet: Fix LogFile sometimes not being set in DesktopCommonAppInstall
When the RegEx matches the first group, the LogFile name was empty

#rb ryan.bazinet

[CL 33229422 by olivier lamothe in ue5-main branch]
2024-04-25 09:29:10 -04:00
jon olick
da218d2bca Instead of passing in AppConfig to Run, store AppConfig in AndroidInstall.
#rb Brendan.Lienau, Chris.Babcock

[CL 33050622 by jon olick in ue5-main branch]
2024-04-17 17:45:03 -04:00
jon olick
c212eb5c54 Use AFS instead of ADB when available.
[CL 33043921 by jon olick in ue5-main branch]
2024-04-17 14:46:09 -04:00
brendan lienau
f0a72e06bf [Gauntlet] Implements new launch flow functions for TargetDeviceDesktopCommon
Also implements overrides for each desktop platform (Win64, WinGDK, Linux, Mac)

#jira UE-201386
#rb David.Harvey, Jerome.Delattre

[CL 32056288 by brendan lienau in ue5-main branch]
2024-03-06 11:47:05 -05:00
brendan lienau
4db3c0ec18 [Gauntlet] Mark Gauntlet builds with whether or not they have the capability to copy additional files. UnrealSession will only copy files if the build supports it
#rb Andrew.Grant

[CL 32032840 by brendan lienau in ue5-main branch]
2024-03-05 15:15:35 -05:00