Commit Graph

81 Commits

Author SHA1 Message Date
andrew grant
85e93f9957 Allow filters for automated tests to be on multiple lines
#jira nojira
#rb na
#lockdown nick.penwarden

#ROBOMERGE-SOURCE: CL 10325873 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v593-10286020)

[CL 10326542 by andrew grant in Main branch]
2019-11-20 17:05:29 -05:00
marc audy
dbcf07a558 Use new inline versions of substring functions
Fix up nearby cases where ESearchCase::CaseSensitive should have been used
#jira
#rnx
#rb

#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 10309793 via CL 10309818
#ROBOMERGE-BOT: (v593-10286020)

[CL 10309932 by marc audy in Main branch]
2019-11-19 13:27:07 -05:00
andrew grant
51bec3d9a1 Moved artifact copying to ParallelFor and added additional logging to help with test timeout (believed to be slow network issues).
#jira UE-81628
#rb na
[CODEREVIEW] josh.engebretson

#ROBOMERGE-SOURCE: CL 9900211 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v558-9892490)

[CL 9900215 by andrew grant in Main branch]
2019-10-31 10:16:22 -04:00
josh engebretson
e961cbc6d6 Add additional logging during EngineTest report generation
#jira UE-81707
#rnx
#rb ben.salem

#ROBOMERGE-SOURCE: CL 9744997 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v542-9736015)

[CL 9745005 by josh engebretson in Main branch]
2019-10-22 09:35:40 -04:00
steve robb
14cab21e0c Deprecating ARRAY_COUNT and changing it to UE_ARRAY_COUNT.
Replicated from CL# 7924370.

#rb none


#ROBOMERGE-OWNER: steve.robb
#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 9279060 via CL 9279063
#ROBOMERGE-BOT: (v443-9013191)

[CL 9279836 by steve robb in Main branch]
2019-09-28 08:19:35 -04:00
Andrew Grant
15284697b3 Switched to curly braces for reporting and parsing test results.
#jira #rb na

[CL 8655294 by Andrew Grant in Main branch]
2019-09-12 11:04:44 -04:00
Andrew Grant
aca0abd590 Fixed issue where tests that timed out weren't written to the log.
Also improved readability of post-test report

#jira #rb na

[CL 8649425 by Andrew Grant in Main branch]
2019-09-11 23:13:25 -04:00
Andrew Grant
4df7819655 Moved low level test settings to a place with no engine dependencies.
#jira UE-74084
#rb none
#fyi ben.marsh,max.chen

[CL 6258941 by Andrew Grant in Main branch]
2019-05-02 15:32:14 -04:00
andrew grant
3849b61f6b Moved automation settings to a common class.
Removed SetTreatWarningsAsErrors from AutomationFramework. Tests should now override TreatLogErrorsAsErrors and TreatLogWarningsAsErrors to control this behavior.

Tidied up some logging and formalized messages emitted by tests.

Emit a Gauntlet-parsed message on test completion

#rb CR'd


#ROBOMERGE-SOURCE: CL 6236960 via CL 6242044

[CL 6242091 by andrew grant in Main branch]
2019-05-01 21:54:37 -04:00
andrew grant
5695c4b8b3 Temporarily removed exit report used by CIS as it turns out some Fortnite tests are actually silently failing :(
#ROBOMERGE-SOURCE: CL 6132799 via CL 6133272

[CL 6174327 by andrew grant in Main branch]
2019-04-30 11:28:21 -04:00
andrew grant
ca5fdf9244 Cleaned up some logging in automation controller
#ROBOMERGE-SOURCE: CL 6115332 via CL 6132874

[CL 6174055 by andrew grant in Main branch]
2019-04-30 11:26:15 -04:00
andrew grant
fc33f5c82a Added support for allowing definition and execution of "groups" of tests using the legacy automation framework.
Groups are defined in DefaultEngine.ini by specifying a groupname and then a set of filters that are used to determine group membership. Groups of tests may then be executed via "RunTests Group:GroupName".

Filters can now also be specified via "RunTests Filter:System+Filter:Editor". This will correctly match filters instead of the old behavior of doing a substring search e.g. SomeNonEditorTest is executed by "RunTests Editor"

Setup some common groups of tests in EngineTest project.

Added TestFilter option to RunAutomatedTests Gauntlet script. Defaults to Project+System

#rb swarm


#ROBOMERGE-SOURCE: CL 6083324 via CL 6088169

[CL 6088218 by andrew grant in Main branch]
2019-04-24 18:33:45 -04:00
andrew grant
b947818c92 Fix duplication of output from test results
https://udn.unrealengine.com/questions/490684/view.html

#jira nojira
#rb none
#lockdown Nick.Penwarden

#ROBOMERGE-SOURCE: CL 5821228 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)

[CL 5821232 by andrew grant in Main branch]
2019-04-10 11:46:47 -04:00
william ewen
6b08c26594 FunctionalTest: Adding ClientDescriptor field to the index.json output reports (in FAutomatedTestPassResults)
#rnx


#ROBOMERGE-SOURCE: CL 5692559 via CL 5695909

[CL 5695947 by william ewen in Main branch]
2019-04-02 18:13:54 -04:00
bob tellez
bd39cccbdb Back out changelist 5686999. Monolithic compile errors
[FYI] William.Ewen


#ROBOMERGE-SOURCE: CL 5690912 via CL 5695359

[CL 5695458 by bob tellez in Main branch]
2019-04-02 18:02:41 -04:00
william ewen
c0932da9c3 FunctionalTest: Adding ClientDescriptor field to the index.json output reports (in FAutomatedTestPassResults)
#rnx


#ROBOMERGE-SOURCE: CL 5686999 via CL 5693258

[CL 5693439 by william ewen in Main branch]
2019-04-02 17:45:04 -04:00
andrew grant
0de4efcf07 Fixed warnings when workers are not ready yet.
Cut down on number of warnings issued.

#jira UE-66856
#rb na


#ROBOMERGE-SOURCE: CL 4925036 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)

[CL 4925037 by andrew grant in Main branch]
2019-02-06 17:43:24 -05:00
ben marsh
2ee0f09118 Fix uninitialized struct member causing warnings when creating DDC on Mac.
#rb none
#jira UE-69436

#ROBOMERGE-SOURCE: CL 4866847 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)

[CL 4867291 by ben marsh in Main branch]
2019-02-01 10:50:19 -05:00
Michael Kirzinger
5f4e2b4396 Copying //UE4/Dev-Online to Dev-Main (//UE4/Dev-Main)
Source: //UE4/Dev-Online 4860295

#rb none

[CL 4860323 by Michael Kirzinger in Main branch]
2019-01-31 15:08:35 -05:00
Ben Marsh
7598af0532 Update copyright notices to 2019.
#rb none
#lockdown Nick.Penwarden

[CL 4662404 by Ben Marsh in Main branch]
2018-12-14 13:41:00 -05:00
Marc Audy
af90b7bcd4 Copying //UE4/Fortnite-Staging to Dev-Main (//UE4/Dev-Main) @ 4395008
#rb
#rnx
#lockdown Nick.Penwarden

[CL 4395058 by Marc Audy in Main branch]
2018-09-25 10:11:35 -04:00
Marc Audy
d90da4ab1a Merge to Dev-Main for 4.20 @ 4090813
#rb
#rnx
#lockdown Nick.Penwarden

[CL 4091081 by Marc Audy in Main branch]
2018-05-23 21:04:31 -04:00
Ben Marsh
75e9f5c691 Print messages when causing the editor to force-quit after running automated tests, and ignore exit codes on errors.
#rb none

[CL 4014962 by Ben Marsh in Dev-Geometry branch]
2018-04-19 20:53:44 -04:00
Matt Kuhlenschmidt
68f4525e1b Merging //UE4/Dev-Main to Dev-Geometry (//UE4/Dev-Geometry)
#rb none

[CL 3889612 by Matt Kuhlenschmidt in Dev-Geometry branch]
2018-02-14 14:13:42 -05:00
Matt Kuhlenschmidt
92eed46d3f Merging //UE4/Dev-Main to Dev-Geometry (//UE4/Dev-Geometry)
#rb none

[CL 3635882 by Matt Kuhlenschmidt in Dev-Geometry branch]
2017-09-11 10:43:35 -04:00