Commit Graph

103 Commits

Author SHA1 Message Date
Marc Audy
bc88b73a29 Merge Release-Engine-Staging to Main @ CL# 15151250
Represents UE4/Main @ 15133763

[CL 15158774 by Marc Audy in ue5-main branch]
2021-01-21 16:22:06 -04:00
geoff evans
6b85337533 Create IterationProfile automated tests to gather raw trace data from Editor
UAT:
Add a new EditorTests runner, existing runner is derelict (will be deleted in upcoming merge from UE4)
BuildGraph is project-agnostic, forked and modified from EngineTest's build graph. Follow up change will try to generalize this logic for future benefit.
Requires specification on the cmdline of project content to use and platform to run on. Project content is specified in DefaultEngine.ini.
EditorTest plugin and log error/warning handling is all handled on the command line to be unobtrusive to production uprojects

EngineTest Plugin:
Add new EditorIterationTests.cpp and a simple test object. This is copied from another test for now for the sake of iteration profiling having its own filter and automation behavior.

AutomationController:
bTreatLogErrorsAsTestErrors was specified in some ini files, but was bugged. Restore this likely deleted config variable and check it when scanning log results

Reverb:
Add Iteration test group

EngineTest:
Add Iteration test group

#rb jerome.delattre, francis.hurteau
#fyi andrew.grant

[CL 14874020 by geoff evans in ue5-main branch]
2020-12-07 17:51:46 -04:00
Arciel Rekman
cb998fe373 Remove unused private fields - Automation
#rb Andrew.Grant
#review-14242502 @Andrew.Grant
#jira none

[CL 14243816 by Arciel Rekman in ue5-main branch]
2020-09-02 09:24:31 -04:00
brooke hubert
48113fc77e Adding EditorFramework to build.cs files
#rnx
#Jira UE-96448
#rb chris.gagnon

[CL 14114839 by brooke hubert in ue5-main branch]
2020-08-14 13:24:16 -04:00
Marc Audy
a7c9001a94 Merging //UE5/Release-Engine-Staging to Main (//UE5/Main) @ 14075166
#rb
#rnx

[CL 14075271 by Marc Audy in ue5-main branch]
2020-08-11 01:36:57 -04:00
Marc Audy
11f5b21210 Merging //UE5/Release-Engine-Staging @ 13752110 to Main (//UE5/Main)
#rnx

[CL 13753156 by Marc Audy in ue5-main branch]
2020-06-23 18:40:00 -04:00
zabir hoque
8c89dae9ea Enable RenderDoc Captures for failing automation tests.
[FYI] Andrew.Grant; Arciel.Rekman;
#rb: none
#jira: UE-91964

#ROBOMERGE-SOURCE: CL 12941114 in //UE4/Release-4.25/... via CL 12941117 via CL 12941124
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v682-12900288)

[CL 12941128 by zabir hoque in Main branch]
2020-04-20 19:57:16 -04:00
andrew grant
add6fd1b3f Added improved logging to diagnose failures in saving screenshots
#jira UE-91964
#rb trivial
[FYI] zabir.hoque
#lockdown cristina.riveron

#ROBOMERGE-SOURCE: CL 12868463 in //UE4/Release-4.25/... via CL 12868468 via CL 12868470
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v681-12776863)

[CL 12868473 by andrew grant in Main branch]
2020-04-16 11:51:35 -04:00
andrew grant
6c1a380c64 Fix issue where new/changed screenshots could not be approved in the editor on some platforms (Mac, Xbox, others?)
The issue was that the code that captures screenshots on a device and sends them was the editor was setting a 'path' variable that represented what a screenshot *would* be saved at on the local system. (The screenshot was not actually saved, it's returned as raw data in the message).

This meant that the editor would try to save for screenshots to a local path that did not exist (e.g. g:\settings\ on xbox) and fail.

The fixes here are -

1) Add some warnings to the code that tries to save screenshots if that fails

2) Change the names of the screenshot functions in AutomationCommon to clarify what they should be used for.

3) Change the 'path' variable in the message to screenshot name.

4) Calculate the path to use for screenshots on the editor.


#tests ran EngineTest on WIn64, PS4, and XboxOne (which now passes).
#jira UE-69399
#rb swarm

#ROBOMERGE-SOURCE: CL 11733277 in //UE4/Release-4.25/... via CL 11733309
#ROBOMERGE-BOT: RELEASE (Release-4.25Plus -> Main) (v656-11643781)

[CL 11733341 by andrew grant in Main branch]
2020-02-28 09:47:55 -05:00
josh engebretson
6b420fdee5 Add initializers for AutomationControllerManager
#rnx
#rb trivial


#ROBOMERGE-SOURCE: CL 11178309 via CL 11178805
#ROBOMERGE-BOT: (v640-11091645)

[CL 11178912 by josh engebretson in Main branch]
2020-01-30 13:13:00 -05:00
josh engebretson
c036ee60ab Fix copyright issue
#ROBOMERGE-SOURCE: CL 11178049 via CL 11178784
#ROBOMERGE-BOT: (v640-11091645)

[CL 11178886 by josh engebretson in Main branch]
2020-01-30 13:12:00 -05:00
josh engebretson
c30553b865 Fix isue with automated test command line delegate
#rb andrew.grant


#ROBOMERGE-SOURCE: CL 11177879 via CL 11178775
#ROBOMERGE-BOT: (v640-11091645)

[CL 11178865 by josh engebretson in Main branch]
2020-01-30 13:11:19 -05:00
josh engebretson
4a03536cec Configuration of AutomationController timings
#rnx
#rb andrew.grant


#ROBOMERGE-SOURCE: CL 10981757 via CL 10981759
#ROBOMERGE-BOT: (v632-10940481)

[CL 10981761 by josh engebretson in Main branch]
2020-01-14 10:36:18 -05:00
sebastien lussier
53fc410b0a Greatly sped up gathering of automation tests in the Test Automation dialog
* Selecting "All Tests" on FN now takes 2-3 seconds instead of minutes (hours?)
#rb patrick.enfedaque


#ROBOMERGE-SOURCE: CL 10941808 via CL 10941810
#ROBOMERGE-BOT: (v632-10940481)

[CL 10941811 by sebastien lussier in Main branch]
2020-01-10 11:27:15 -05:00
andrew grant
fa6d2fd0ee Fix for automation results being added to an anonymous category in the editors log window.
Also tidied up output formatiing a little.

#jira UE-86291
#rb na
#lockdown cristina.riveron

#ROBOMERGE-SOURCE: CL 10883961 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v626-10872990)

[CL 10883962 by andrew grant in Main branch]
2020-01-06 16:38:25 -05:00
ryan durand
471d972e62 Updating copyright for Engine Developer.
#rnx
#rb none


#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)

[CL 10870584 by ryan durand in Main branch]
2019-12-26 15:32:37 -05:00
andrew grant
85e93f9957 Allow filters for automated tests to be on multiple lines
#jira nojira
#rb na
#lockdown nick.penwarden

#ROBOMERGE-SOURCE: CL 10325873 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v593-10286020)

[CL 10326542 by andrew grant in Main branch]
2019-11-20 17:05:29 -05:00
marc audy
dbcf07a558 Use new inline versions of substring functions
Fix up nearby cases where ESearchCase::CaseSensitive should have been used
#jira
#rnx
#rb

#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 10309793 via CL 10309818
#ROBOMERGE-BOT: (v593-10286020)

[CL 10309932 by marc audy in Main branch]
2019-11-19 13:27:07 -05:00
andrew grant
51bec3d9a1 Moved artifact copying to ParallelFor and added additional logging to help with test timeout (believed to be slow network issues).
#jira UE-81628
#rb na
[CODEREVIEW] josh.engebretson

#ROBOMERGE-SOURCE: CL 9900211 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v558-9892490)

[CL 9900215 by andrew grant in Main branch]
2019-10-31 10:16:22 -04:00
josh engebretson
e961cbc6d6 Add additional logging during EngineTest report generation
#jira UE-81707
#rnx
#rb ben.salem

#ROBOMERGE-SOURCE: CL 9744997 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v542-9736015)

[CL 9745005 by josh engebretson in Main branch]
2019-10-22 09:35:40 -04:00
steve robb
14cab21e0c Deprecating ARRAY_COUNT and changing it to UE_ARRAY_COUNT.
Replicated from CL# 7924370.

#rb none


#ROBOMERGE-OWNER: steve.robb
#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 9279060 via CL 9279063
#ROBOMERGE-BOT: (v443-9013191)

[CL 9279836 by steve robb in Main branch]
2019-09-28 08:19:35 -04:00
Andrew Grant
15284697b3 Switched to curly braces for reporting and parsing test results.
#jira #rb na

[CL 8655294 by Andrew Grant in Main branch]
2019-09-12 11:04:44 -04:00
Andrew Grant
aca0abd590 Fixed issue where tests that timed out weren't written to the log.
Also improved readability of post-test report

#jira #rb na

[CL 8649425 by Andrew Grant in Main branch]
2019-09-11 23:13:25 -04:00
Andrew Grant
b9c7c16943 Missed file from 6258941
#jira #rb na

[CL 6258972 by Andrew Grant in Main branch]
2019-05-02 15:34:13 -04:00
Andrew Grant
4df7819655 Moved low level test settings to a place with no engine dependencies.
#jira UE-74084
#rb none
#fyi ben.marsh,max.chen

[CL 6258941 by Andrew Grant in Main branch]
2019-05-02 15:32:14 -04:00