Commit Graph

157 Commits

Author SHA1 Message Date
Jerome Delattre
590093ed9b Always take the last member of the string for build command
+ add more arguments to filter out

#preflight 6373abd81c114bec05ac2458
#rnx
#jira none
#rb Chris.Constantinescu

[CL 23159710 by Jerome Delattre in ue5-main branch]
2022-11-16 14:47:44 -05:00
jerome delattre
3c4ffdb8f4 Use Structured Logging in Gauntlet
#jira UE-163471
#preflight 63700c1e0c74adb48bb9a5c9
#rb J.Baumgartner, Erik.Knapik, Chris.Constantinescu, Josh.Engebretson

[CL 23112357 by jerome delattre in ue5-main branch]
2022-11-12 18:42:11 -05:00
eric knapik
6daa77dced #jira: none
Typo fixup

[CL 23091032 by eric knapik in ue5-main branch]
2022-11-10 19:58:12 -05:00
Josh Engebretson
85b49aca06 Horde: Add version to Gauntlet horde reports, add some basic test information to simple reports for new backend
#jira none
#rnx
#rb Jerome.Delattre
#preflight 636a640af56cab38c7242313

[CL 23053411 by Josh Engebretson in ue5-main branch]
2022-11-09 08:55:04 -05:00
eric knapik
b095d01d71 #jira: none
Undo CL: 22849610
This caused issues with replay testing as null artifacts were not correctly being saved.
Adjusting the FunctionalTest so it runs then generates a blank artifact.
overriding the GetTestSummary header when we know no test will be ran.

#preflight 6361464763608aee36f71c87

[CL 22890128 by eric knapik in ue5-main branch]
2022-11-01 16:01:26 -04:00
brendan lienau
c4f555898b [Automation] Gives the ability to specify a "-staged" param which will append a Loose BuildFlag requirement if valid
#rb j.baumgartner

[CL 22888727 by brendan lienau in ue5-main branch]
2022-11-01 15:11:51 -04:00
eric knapik
eb7ece625e #jira: UE-168744
Add null check on InLog

[REVIEW] Jerome.Delattre
#preflight 636012cc561635187863caf4

[CL 22875474 by eric knapik in ue5-main branch]
2022-10-31 20:03:11 -04:00
eric knapik
79662b0c11 #jira: FORT-507969
Disabling many creative tests that don't work
Adding new ContextFunctionalTest to look at all of the previous files that have been modified and find the tests that best match the modified files

Added additional core function for getting the referencers for a set of assets up to a specified search depth.

Added the ContextFunctionalTest gauntlet job so we can run this in horde
example launching method:

[REVIEW] [at]Gary.Yuan [at]Tim.Kennedy [at]Cory.Kolek
#preflight 635c7dc41803be35c7886bbe

[CL 22850798 by eric knapik in ue5-main branch]
2022-10-29 03:01:13 -04:00
jerome delattre
53590855ec Gauntlet - Clear RolesToInstalls only if it was initialized
#jira UE-166161
#rnx
#rb Willian.Ewen
#preflight 6356f53d3d41a4455a640c80

[CL 22787022 by jerome delattre in ue5-main branch]
2022-10-26 15:29:16 -04:00
jerome delattre
d25695567d Get Editor role based on Config setting (unify Editor and CookedEditor)
#rnx
#rb Eric.Knapik
#preflight 6352ad010313c249742d11ae

[CL 22711554 by jerome delattre in ue5-main branch]
2022-10-22 14:43:21 -04:00
eric knapik
0ffc2e54b8 #jira: none
Adding quotes as some tests can have a . in them

[CL 22634733 by eric knapik in ue5-main branch]
2022-10-19 15:11:50 -04:00
eric knapik
c1544b1a9c #jira: FORT-526894
Add missing -test= to how to run locally text

[CL 22606078 by eric knapik in ue5-main branch]
2022-10-18 13:59:37 -04:00
eric knapik
26df6dea74 #jira: FORT-477143, FORT-448935
Have gauntlet display the command a test was ran with
Add server output to the horde log so errors can be easier identified from a webpage.
Fixing a typo with Text which should have been Test

#preflight 63408864587d6afac86da474
#rb: Gary.Yuan
https://p4-swarm.epicgames.net/reviews/22408413/

[CL 22451857 by eric knapik in ue5-main branch]
2022-10-11 02:54:50 -04:00
jerome delattre
6e8068c544 Add argument to setup Module Role
+ Enforce using a the CustomModuleRoles argument with CookedEditor in BuildAndTestProject buildgraph

#jira UE-165358
#rnx
#preflight 6335ed6ed165c45137b2eaf6
#rb Chris.Constinescu

[CL 22284248 by jerome delattre in ue5-main branch]
2022-09-30 19:25:03 -04:00
william ewen
f9530b1ad3 Gauntlet: Make sure we don't erroneously try to reuse an install on a previous device if we had trouble launching on that device, or if we released devices at any point
#rb josh.engebretson

[CL 22264062 by william ewen in ue5-main branch]
2022-09-30 01:36:51 -04:00
j baumgartner
df850050eb Compilation fix
#rb william.ewen

[CL 22148167 by j baumgartner in ue5-main branch]
2022-09-22 18:50:21 -04:00
william ewen
fa6b069775 Gauntlet: Make it so multi-pass tests will retain devices between each run
#rb Josh.Engebretson
[FYI] Andrew.Grant

[CL 22147439 by william ewen in ue5-main branch]
2022-09-22 18:17:10 -04:00
brandon schaefer
e08d29da57 Disable splash screen when running unattended
EngineTest simply does not have a splash screen and we warn on Linux. Lets just disable it for these tests

#jira UE-153920
#rb Josh.Adams
#preflight N/A
[FYI] Robert.Seiver

[CL 22042330 by brandon schaefer in ue5-main branch]
2022-09-15 20:02:08 -04:00
chris constantinescu
ddf55c6ac2 Retry save role artifacts once with device reboot on exception containing message "A retry should be performed"
#jira UE-161735
#rb Jerome.Delattre
#preflight skip

[CL 21896605 by chris constantinescu in ue5-main branch]
2022-09-08 12:50:25 -04:00
ben salem
4a662bbbac Fix Gauntlet throwing an InitializationFailed error when a process fails to log in.
#rb brendan.lienau, j.baumgartner

[CL 21864920 by ben salem in ue5-main branch]
2022-09-07 18:12:06 -04:00
chris constantinescu
b604e668e2 Test runner:
- sleep on init to allow for sync w/ Gauntlet for Xbox platforms
- remove unused --base-global-module #fyi Stephen.Ma
Gauntlet:
- support low level tests build for different configurations (debug etc)
- readable command line arguments (-r to --reporter)
- allow argument names containing "-" e.g. --filenames-as-tags
#preflight 630536b1c00af5e2943540a1
#rb Jerome.Delattre

[CL 21550045 by chris constantinescu in ue5-main branch]
2022-08-24 15:53:16 -04:00
andrew grant
4855221f20 Add logging to launch of tests in UnrealSession that display the role type
#preflight https://horde.devtools.epicgames.com/log/62f6b4c4ad3bd8ad648b4107

#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21364065 via CL 21387057 via CL 21389907 via CL 21390632
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v975-21357124)

[CL 21392562 by andrew grant in ue5-main branch]
2022-08-15 15:50:44 -04:00
andrew grant
02e3ed342d When looking for a Gauntlet exit code check for -gauntlet= and not -gauntlet. It's the former that indicates there will be code in the process that is orchestrating the test.
#rb swarm
[REVIEW] [at]eric.knapik [at]daniel.lamb
#preflight https://horde.devtools.epicgames.com/job/62f54b3cbd746abb9987728c

#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21343066 via CL 21350278 via CL 21350428 via CL 21350452
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21350672 by andrew grant in ue5-main branch]
2022-08-11 23:42:50 -04:00
jerome delattre
2652ce1b3f Avoid enforcing Horde Simple Report to all UnrealTestNode.
#preflight 62c8accfb04c766648a41525
#rb none

#ROBOMERGE-AUTHOR: jerome.delattre
#ROBOMERGE-SOURCE: CL 21017400 via CL 21017405 via CL 21017409
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21023880 by jerome delattre in ue5-main branch]
2022-07-09 14:05:05 -04:00
chris constantinescu
1b1fb1ce98 Change Tcp Messaging log level to Verbose on Horde
#rb Jerome.Delattre
#preflight 62bccb3fd94b57687c9e037f

[CL 20886061 by chris constantinescu in ue5-main branch]
2022-06-29 19:07:32 -04:00