Commit Graph

474 Commits

Author SHA1 Message Date
jerome delattre
53590855ec Gauntlet - Clear RolesToInstalls only if it was initialized
#jira UE-166161
#rnx
#rb Willian.Ewen
#preflight 6356f53d3d41a4455a640c80

[CL 22787022 by jerome delattre in ue5-main branch]
2022-10-26 15:29:16 -04:00
jerome delattre
d25695567d Get Editor role based on Config setting (unify Editor and CookedEditor)
#rnx
#rb Eric.Knapik
#preflight 6352ad010313c249742d11ae

[CL 22711554 by jerome delattre in ue5-main branch]
2022-10-22 14:43:21 -04:00
eric knapik
0ffc2e54b8 #jira: none
Adding quotes as some tests can have a . in them

[CL 22634733 by eric knapik in ue5-main branch]
2022-10-19 15:11:50 -04:00
dmitrii ubirailo
069e2d1532 Supported defining alternative folder for UEFN by setting environemt variable UE_BUILDS_PATH.
[REVIEW] Anna.Fukutome, Eric.Knapik
#preflight skip

[CL 22619920 by dmitrii ubirailo in ue5-main branch]
2022-10-19 03:05:28 -04:00
eric knapik
c1544b1a9c #jira: FORT-526894
Add missing -test= to how to run locally text

[CL 22606078 by eric knapik in ue5-main branch]
2022-10-18 13:59:37 -04:00
j baumgartner
bd4ac13fc4 [Automation] Add missing logic to handle the new -perfmodel= commandline to gauntlet.
#rb josh.engebretson, ben.salem
#tests tested running a few tests with perfmodel and perfspec to make sure both functionalities worked
#rnx

[CL 22546730 by j baumgartner in ue5-main branch]
2022-10-14 22:19:53 -04:00
eric knapik
26df6dea74 #jira: FORT-477143, FORT-448935
Have gauntlet display the command a test was ran with
Add server output to the horde log so errors can be easier identified from a webpage.
Fixing a typo with Text which should have been Test

#preflight 63408864587d6afac86da474
#rb: Gary.Yuan
https://p4-swarm.epicgames.net/reviews/22408413/

[CL 22451857 by eric knapik in ue5-main branch]
2022-10-11 02:54:50 -04:00
gary yuan
3ddd4e57c7 Marking up more parameter help attributes on Gauntlet commands
#rb eric.knapik
#preflight 633cc7f3393e657831cef406

[CL 22366405 by gary yuan in ue5-main branch]
2022-10-05 19:16:50 -04:00
gary yuan
676c1e7db2 Marking up Gauntlet classes with parameter help attributes
#rb Tim.Kennedy
#preflight 633711f15c2225fe5f78f7b7

[CL 22286636 by gary yuan in ue5-main branch]
2022-09-30 23:10:15 -04:00
jerome delattre
6e8068c544 Add argument to setup Module Role
+ Enforce using a the CustomModuleRoles argument with CookedEditor in BuildAndTestProject buildgraph

#jira UE-165358
#rnx
#preflight 6335ed6ed165c45137b2eaf6
#rb Chris.Constinescu

[CL 22284248 by jerome delattre in ue5-main branch]
2022-09-30 19:25:03 -04:00
william ewen
f9530b1ad3 Gauntlet: Make sure we don't erroneously try to reuse an install on a previous device if we had trouble launching on that device, or if we released devices at any point
#rb josh.engebretson

[CL 22264062 by william ewen in ue5-main branch]
2022-09-30 01:36:51 -04:00
william ewen
19b11fcb3e Gauntlet: Make DeviceReservations forget problem devices between passes, so that the DeviceService is authoritative
#rb josh.engebretson

[CL 22203062 by william ewen in ue5-main branch]
2022-09-27 06:57:20 -04:00
j baumgartner
df850050eb Compilation fix
#rb william.ewen

[CL 22148167 by j baumgartner in ue5-main branch]
2022-09-22 18:50:21 -04:00
william ewen
fa6b069775 Gauntlet: Make it so multi-pass tests will retain devices between each run
#rb Josh.Engebretson
[FYI] Andrew.Grant

[CL 22147439 by william ewen in ue5-main branch]
2022-09-22 18:17:10 -04:00
brandon schaefer
e08d29da57 Disable splash screen when running unattended
EngineTest simply does not have a splash screen and we warn on Linux. Lets just disable it for these tests

#jira UE-153920
#rb Josh.Adams
#preflight N/A
[FYI] Robert.Seiver

[CL 22042330 by brandon schaefer in ue5-main branch]
2022-09-15 20:02:08 -04:00
chris constantinescu
ddf55c6ac2 Retry save role artifacts once with device reboot on exception containing message "A retry should be performed"
#jira UE-161735
#rb Jerome.Delattre
#preflight skip

[CL 21896605 by chris constantinescu in ue5-main branch]
2022-09-08 12:50:25 -04:00
ben salem
4a662bbbac Fix Gauntlet throwing an InitializationFailed error when a process fails to log in.
#rb brendan.lienau, j.baumgartner

[CL 21864920 by ben salem in ue5-main branch]
2022-09-07 18:12:06 -04:00
Jerome Delattre
040633abd1 Fix Cooked Editor exported path in Windows
+ set CookedEditor role in Gauntlet UE tests when -cookededitor switch is used

#jira none
#rb Nick.bullard
#preflight 631269c6a20b67673b862af4
#rnx

[CL 21849523 by Jerome Delattre in ue5-main branch]
2022-09-07 09:51:36 -04:00
chris constantinescu
b604e668e2 Test runner:
- sleep on init to allow for sync w/ Gauntlet for Xbox platforms
- remove unused --base-global-module #fyi Stephen.Ma
Gauntlet:
- support low level tests build for different configurations (debug etc)
- readable command line arguments (-r to --reporter)
- allow argument names containing "-" e.g. --filenames-as-tags
#preflight 630536b1c00af5e2943540a1
#rb Jerome.Delattre

[CL 21550045 by chris constantinescu in ue5-main branch]
2022-08-24 15:53:16 -04:00
chris constantinescu
f5123e1e5f Re-throw if not a "retry" exception.
#rb Jerome.Delattre
#preflight skip

[CL 21457802 by chris constantinescu in ue5-main branch]
2022-08-19 11:10:00 -04:00
chris constantinescu
5f866f13cf Retry various file system operations that could fail for PS5 file system.
#jira UE-160038
#preflight 62fe598e3d3fb466b22e8b43
#rb Jerome.Delattre

[CL 21445586 by chris constantinescu in ue5-main branch]
2022-08-18 13:38:03 -04:00
andrew grant
4855221f20 Add logging to launch of tests in UnrealSession that display the role type
#preflight https://horde.devtools.epicgames.com/log/62f6b4c4ad3bd8ad648b4107

#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21364065 via CL 21387057 via CL 21389907 via CL 21390632
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v975-21357124)

[CL 21392562 by andrew grant in ue5-main branch]
2022-08-15 15:50:44 -04:00
eric mcdaniel
8ccb89a3c8 Xbox: Increase initial wait time on PLM test to account for long startup times experienced by XB1 on Horde
- PLM test is failing sporadically on Horde due to the first suspend attempt being made before any frames have been displayed
    - locally I see ~5 secs before the app is displaying frames but this is ~20 more on Horde for some runs
    - even successful runs are nearing the current timeout (30s) on XB1
    - unclear why some Horde startups are so slow so just increase the initial wait to 45 secs to give us some headroom

#rb David.Harvey
#jira UE-159810
#preflight 62fa6f24ad3bd8ad640f8a99

[CL 21387897 by eric mcdaniel in ue5-main branch]
2022-08-15 12:14:20 -04:00
andrew grant
02e3ed342d When looking for a Gauntlet exit code check for -gauntlet= and not -gauntlet. It's the former that indicates there will be code in the process that is orchestrating the test.
#rb swarm
[REVIEW] [at]eric.knapik [at]daniel.lamb
#preflight https://horde.devtools.epicgames.com/job/62f54b3cbd746abb9987728c

#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21343066 via CL 21350278 via CL 21350428 via CL 21350452
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21350672 by andrew grant in ue5-main branch]
2022-08-11 23:42:50 -04:00
Joe Conley
604cae73eb #jira UE-147963 Boot test for Hololens
Support for Hololens gauntlet automation launching to device
#preflight 62f3f9a0b43215cf6c2d98ac

[CL 21324460 by Joe Conley in ue5-main branch]
2022-08-10 15:47:10 -04:00