Commit Graph

94 Commits

Author SHA1 Message Date
josh engebretson
6b420fdee5 Add initializers for AutomationControllerManager
#rnx
#rb trivial


#ROBOMERGE-SOURCE: CL 11178309 via CL 11178805
#ROBOMERGE-BOT: (v640-11091645)

[CL 11178912 by josh engebretson in Main branch]
2020-01-30 13:13:00 -05:00
josh engebretson
c036ee60ab Fix copyright issue
#ROBOMERGE-SOURCE: CL 11178049 via CL 11178784
#ROBOMERGE-BOT: (v640-11091645)

[CL 11178886 by josh engebretson in Main branch]
2020-01-30 13:12:00 -05:00
josh engebretson
c30553b865 Fix isue with automated test command line delegate
#rb andrew.grant


#ROBOMERGE-SOURCE: CL 11177879 via CL 11178775
#ROBOMERGE-BOT: (v640-11091645)

[CL 11178865 by josh engebretson in Main branch]
2020-01-30 13:11:19 -05:00
josh engebretson
4a03536cec Configuration of AutomationController timings
#rnx
#rb andrew.grant


#ROBOMERGE-SOURCE: CL 10981757 via CL 10981759
#ROBOMERGE-BOT: (v632-10940481)

[CL 10981761 by josh engebretson in Main branch]
2020-01-14 10:36:18 -05:00
sebastien lussier
53fc410b0a Greatly sped up gathering of automation tests in the Test Automation dialog
* Selecting "All Tests" on FN now takes 2-3 seconds instead of minutes (hours?)
#rb patrick.enfedaque


#ROBOMERGE-SOURCE: CL 10941808 via CL 10941810
#ROBOMERGE-BOT: (v632-10940481)

[CL 10941811 by sebastien lussier in Main branch]
2020-01-10 11:27:15 -05:00
andrew grant
fa6d2fd0ee Fix for automation results being added to an anonymous category in the editors log window.
Also tidied up output formatiing a little.

#jira UE-86291
#rb na
#lockdown cristina.riveron

#ROBOMERGE-SOURCE: CL 10883961 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v626-10872990)

[CL 10883962 by andrew grant in Main branch]
2020-01-06 16:38:25 -05:00
ryan durand
471d972e62 Updating copyright for Engine Developer.
#rnx
#rb none


#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)

[CL 10870584 by ryan durand in Main branch]
2019-12-26 15:32:37 -05:00
andrew grant
85e93f9957 Allow filters for automated tests to be on multiple lines
#jira nojira
#rb na
#lockdown nick.penwarden

#ROBOMERGE-SOURCE: CL 10325873 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v593-10286020)

[CL 10326542 by andrew grant in Main branch]
2019-11-20 17:05:29 -05:00
marc audy
dbcf07a558 Use new inline versions of substring functions
Fix up nearby cases where ESearchCase::CaseSensitive should have been used
#jira
#rnx
#rb

#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 10309793 via CL 10309818
#ROBOMERGE-BOT: (v593-10286020)

[CL 10309932 by marc audy in Main branch]
2019-11-19 13:27:07 -05:00
andrew grant
51bec3d9a1 Moved artifact copying to ParallelFor and added additional logging to help with test timeout (believed to be slow network issues).
#jira UE-81628
#rb na
[CODEREVIEW] josh.engebretson

#ROBOMERGE-SOURCE: CL 9900211 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v558-9892490)

[CL 9900215 by andrew grant in Main branch]
2019-10-31 10:16:22 -04:00
josh engebretson
e961cbc6d6 Add additional logging during EngineTest report generation
#jira UE-81707
#rnx
#rb ben.salem

#ROBOMERGE-SOURCE: CL 9744997 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v542-9736015)

[CL 9745005 by josh engebretson in Main branch]
2019-10-22 09:35:40 -04:00
steve robb
14cab21e0c Deprecating ARRAY_COUNT and changing it to UE_ARRAY_COUNT.
Replicated from CL# 7924370.

#rb none


#ROBOMERGE-OWNER: steve.robb
#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 9279060 via CL 9279063
#ROBOMERGE-BOT: (v443-9013191)

[CL 9279836 by steve robb in Main branch]
2019-09-28 08:19:35 -04:00
Andrew Grant
15284697b3 Switched to curly braces for reporting and parsing test results.
#jira #rb na

[CL 8655294 by Andrew Grant in Main branch]
2019-09-12 11:04:44 -04:00
Andrew Grant
aca0abd590 Fixed issue where tests that timed out weren't written to the log.
Also improved readability of post-test report

#jira #rb na

[CL 8649425 by Andrew Grant in Main branch]
2019-09-11 23:13:25 -04:00
Andrew Grant
b9c7c16943 Missed file from 6258941
#jira #rb na

[CL 6258972 by Andrew Grant in Main branch]
2019-05-02 15:34:13 -04:00
Andrew Grant
4df7819655 Moved low level test settings to a place with no engine dependencies.
#jira UE-74084
#rb none
#fyi ben.marsh,max.chen

[CL 6258941 by Andrew Grant in Main branch]
2019-05-02 15:32:14 -04:00
andrew grant
3849b61f6b Moved automation settings to a common class.
Removed SetTreatWarningsAsErrors from AutomationFramework. Tests should now override TreatLogErrorsAsErrors and TreatLogWarningsAsErrors to control this behavior.

Tidied up some logging and formalized messages emitted by tests.

Emit a Gauntlet-parsed message on test completion

#rb CR'd


#ROBOMERGE-SOURCE: CL 6236960 via CL 6242044

[CL 6242091 by andrew grant in Main branch]
2019-05-01 21:54:37 -04:00
andrew grant
5695c4b8b3 Temporarily removed exit report used by CIS as it turns out some Fortnite tests are actually silently failing :(
#ROBOMERGE-SOURCE: CL 6132799 via CL 6133272

[CL 6174327 by andrew grant in Main branch]
2019-04-30 11:28:21 -04:00
andrew grant
ca5fdf9244 Cleaned up some logging in automation controller
#ROBOMERGE-SOURCE: CL 6115332 via CL 6132874

[CL 6174055 by andrew grant in Main branch]
2019-04-30 11:26:15 -04:00
andrew grant
fc33f5c82a Added support for allowing definition and execution of "groups" of tests using the legacy automation framework.
Groups are defined in DefaultEngine.ini by specifying a groupname and then a set of filters that are used to determine group membership. Groups of tests may then be executed via "RunTests Group:GroupName".

Filters can now also be specified via "RunTests Filter:System+Filter:Editor". This will correctly match filters instead of the old behavior of doing a substring search e.g. SomeNonEditorTest is executed by "RunTests Editor"

Setup some common groups of tests in EngineTest project.

Added TestFilter option to RunAutomatedTests Gauntlet script. Defaults to Project+System

#rb swarm


#ROBOMERGE-SOURCE: CL 6083324 via CL 6088169

[CL 6088218 by andrew grant in Main branch]
2019-04-24 18:33:45 -04:00
andrew grant
b947818c92 Fix duplication of output from test results
https://udn.unrealengine.com/questions/490684/view.html

#jira nojira
#rb none
#lockdown Nick.Penwarden

#ROBOMERGE-SOURCE: CL 5821228 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)

[CL 5821232 by andrew grant in Main branch]
2019-04-10 11:46:47 -04:00
william ewen
6b08c26594 FunctionalTest: Adding ClientDescriptor field to the index.json output reports (in FAutomatedTestPassResults)
#rnx


#ROBOMERGE-SOURCE: CL 5692559 via CL 5695909

[CL 5695947 by william ewen in Main branch]
2019-04-02 18:13:54 -04:00
bob tellez
bd39cccbdb Back out changelist 5686999. Monolithic compile errors
[FYI] William.Ewen


#ROBOMERGE-SOURCE: CL 5690912 via CL 5695359

[CL 5695458 by bob tellez in Main branch]
2019-04-02 18:02:41 -04:00
william ewen
c0932da9c3 FunctionalTest: Adding ClientDescriptor field to the index.json output reports (in FAutomatedTestPassResults)
#rnx


#ROBOMERGE-SOURCE: CL 5686999 via CL 5693258

[CL 5693439 by william ewen in Main branch]
2019-04-02 17:45:04 -04:00
andrew grant
0de4efcf07 Fixed warnings when workers are not ready yet.
Cut down on number of warnings issued.

#jira UE-66856
#rb na


#ROBOMERGE-SOURCE: CL 4925036 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)

[CL 4925037 by andrew grant in Main branch]
2019-02-06 17:43:24 -05:00