Commit Graph

559 Commits

Author SHA1 Message Date
william ewen
203f332ef6 Adding -Packaged param to RunUnreal
#rb Gary.Yuan
#rb Adolf.Stary
#rb Story.Martin

[CL 26296902 by william ewen in ue5-main branch]
2023-06-28 16:00:05 -04:00
jerome delattre
51f6f270aa Force process output to be logged as verbose log level so it does not trigger the event parser
#rnx
#rb Eric.McDaniel

[CL 26217902 by jerome delattre in ue5-main branch]
2023-06-23 16:24:18 -04:00
tim kennedy
199e463f10 * Removed FortniteRpcTestNode completely, moving the boilerplate test setup to RpcTargetRegistry and RpcExecutor
* Added new class RpcSynchronousTestExecutor - this class allows for a test logic thread to be configured and run where Test Logic can be executed in a synchronus manner. You can use blocking calls and loops here while not interfering with Gauntlet's main loop.
* Added new interface ISynchronousTestLogic to assist with the implementation of RpcSychronousExecutor in Gauntlet tests.
* Added template specialization to FFortBaseExternalRpcHttpResponse due to bool/FString overload resolution errors and updated dependent files.
* Updated UEFNCloseEditor test to use RpcLibrary
* Added ExitEditor RPC (UE + Gauntlet)
* Added in outgoing messages from processes to indicate when ready to begin the test
* Added logic to automatically poll for ready instead of waiting for a message if we're on Xbox since it can't send outbound messages to Gauntlet.
To test locally:
RunUAT.bat RunFortTests -build=editor -tests=UEFNCloseEditor

[CL 26188198 by tim kennedy in ue5-main branch]
2023-06-22 14:12:59 -04:00
brendan lienau
322c5ee790 [Gauntlet] Handle log writing exception
#rnx
#rb trivial

[CL 26096331 by brendan lienau in ue5-main branch]
2023-06-19 13:37:44 -04:00
brendan lienau
2795e5bb2e [Gauntlet] Prevent OOM error when writing log files of substantial size
Prevent log files > 1gb in size from being written.
#rb Clayton.Langford
#rnx

[CL 26091998 by brendan lienau in ue5-main branch]
2023-06-19 11:42:47 -04:00
Jerome Delattre
b47d6a8799 Gauntlet - Add extra context when failing on finding build target
#rnx
#rb trivial

[CL 26013396 by Jerome Delattre in ue5-main branch]
2023-06-15 10:14:22 -04:00
brendan lienau
f78a6043d2 [Gauntlet] Improve clarity for logging in DevicePool creation warnings
This should yield a call stack showing the explicit reason for failure to help track down intermittent issues.
#rb adolf.stary

[CL 25997394 by brendan lienau in ue5-main branch]
2023-06-14 17:43:59 -04:00
tyler staples
be7ecefac8 CitySample - Fix csvs not being found on some platforms.
+ Use PathUtils.FindRelevantPath to search for the FPSChartStats folder within the artifacts dir. Then copy out the newest csv each iteration.
+ Remove deletion of saved dir between iterations as it's no longer needed since we grab the latest csv file.
+ Pass in the build name as it's not reliably set in Gauntlet on all platforms.

#jira UE-186796
#rnx
#rb clayton.langford

[CL 25945245 by tyler staples in ue5-main branch]
2023-06-12 22:20:58 -04:00
mikalai sukhikh
bec1082ff3 Gauntlet - UE.Networking test is not bubbling up error from gauntlet controller
#jira UE-184497
#rb adam.miezianko, chris.constantinescu, jerome.delattre

[CL 25888575 by mikalai sukhikh in ue5-main branch]
2023-06-09 03:43:49 -04:00
Jerome Delattre
02619c779c Raise the exit reason as an error when a role is failing
+ Use the Main role for test summary context.

#rnx
#rb trivial

[CL 25852309 by Jerome Delattre in ue5-main branch]
2023-06-07 14:29:49 -04:00
tyler staples
de803ac1a6 Gauntlet - Add missing copyright notice.
#rnx

[CL 25824781 by tyler staples in ue5-main branch]
2023-06-06 15:12:33 -04:00
tyler staples
6a7f8b3936 Gauntlet - Move PathUtils into the Gauntlet project.
Make these utilities accessible for engine projects.

#rnx
[REVIEW] clayton.langford,brendan.lienau
#rb clayton.langford

[CL 25824160 by tyler staples in ue5-main branch]
2023-06-06 14:58:17 -04:00
Jerome Delattre
625c06620f Use -VerifyLogin when running UE.Automation and UE.BootTest test on clients (2nd try)
#rnx
#rb Chris.constantinescu, Brendan.Lienau

[CL 25817958 by Jerome Delattre in ue5-main branch]
2023-06-06 11:52:09 -04:00
Jerome Delattre
583c1cc9fc [BackOut] changelist 25799020
Use -VerifyLogin when running UE.Automation and UE.BootTest test on clients

#rnx

[CL 25799486 by Jerome Delattre in ue5-main branch]
2023-06-05 14:49:37 -04:00
Jerome Delattre
9ce5e0d787 Report on device reservation also when test fails to start and no retry is left
#jira none
#rnx
#rb Chris.Constantinescu

[CL 25799072 by Jerome Delattre in ue5-main branch]
2023-06-05 14:32:35 -04:00
Jerome Delattre
a94a4aac2e Use -VerifyLogin when running UE.Automation and UE.BootTest test on clients
#rnx
#rb Chris.constantinescu

[CL 25799020 by Jerome Delattre in ue5-main branch]
2023-06-05 14:31:30 -04:00
brendan lienau
24ea396fea [Gauntlet] If an unreal process produces a malformed log, don't skip saving all artifacts when throwing an encoding exception
#rb j.baumgartner

[CL 25627533 by brendan lienau in ue5-main branch]
2023-05-25 16:01:47 -04:00
Jerome Delattre
ed09a0800a Don't dereference ReservationStates on successful reservation
#jira UE-185763
#rnx
#preflight 6462300baa3c584c0bf14798
#rb trivial

[CL 25472748 by Jerome Delattre in ue5-main branch]
2023-05-15 09:47:04 -04:00
Jerome Delattre
0cb33a3dc4 Improve Gauntlet warning message during Device resource starvation
#jira UE-177435
#preflight 64357e454c0cdaea09db2859
#rnx
#rb Clayton.Langford, Chris.Constantinescu, Josh.Engebretson

[CL 25342255 by Jerome Delattre in ue5-main branch]
2023-05-04 16:48:26 -04:00
brendan lienau
1ce61dd21e [Automation] Passthrough BUILDMACHINE to relevant processes when running on a buildmachine.
#rb clayton.langford
#preflight skip

[CL 25339767 by brendan lienau in ue5-main branch]
2023-05-04 14:51:53 -04:00
chris constantinescu
e003cfd4a1 Use Gauntlet to run low level tests instead of BG's Spawn, move LowLevelTests.xml to public folder and remove all restricted platform names
#rb Jerome.Delattre
#preflight 6452abfd4d593c0b422b5d9e

[CL 25335483 by chris constantinescu in ue5-main branch]
2023-05-04 11:05:46 -04:00
matt peters
8441ee518b Change PerfEditorPIETest to use an isolated warming pass and to use a cached asset registry.
#rb Zousar.Shaker
#rnx
#preflight 645173590722d4f6a2aa494d

[CL 25310352 by matt peters in ue5-main branch]
2023-05-02 19:02:34 -04:00
joe barnes
6d239b4ff5 * Ensure any processes (TM2, NintendoSDKDaemon, RunOnTarget) started by Switch SDK utilities (ControlTarget, RunOnTarget) are shut down when we're done.
* Create UsedPlatform HashSet as we add target devices in case their unregistered before we get to dispose().

#rb michael.wanderson
[FYI] [at]brian.white, [at]michael.wanderson

[CL 25241522 by joe barnes in ue5-main branch]
2023-04-28 17:10:52 -04:00
gary yuan
b8ed6fcd22 Gauntlet fix Regex in GetTestExitCode for RequestExitWithStatus
#rb
[REVIEW] eric.knapik
#preflight 644acd52877716c8787bfabd

[CL 25238032 by gary yuan in ue5-main branch]
2023-04-28 15:34:37 -04:00
Jerome Delattre
352db4f4b5 Gauntlet - Use the most appropriate Role to represent UnrealTestNode context
#preflight 64493b2a1150e908d0812ebc
#rnx
#jira none
#rb trivial

[CL 25199447 by Jerome Delattre in ue5-main branch]
2023-04-26 11:39:00 -04:00