Commit Graph

152 Commits

Author SHA1 Message Date
bryan sefcik
a3dddc6630 Pass 1 on Developer include fixes:
Removed redundant private include paths from build.cs files.
Fixed include paths to be relative to the private or public folders.
Hid or removed includes that reached into other private module folders.
Updated PublicInclude paths when necessary.

#jira
#preflight 631e281694758d0bf2ea1399

[CL 21960082 by bryan sefcik in ue5-main branch]
2022-09-11 18:32:18 -04:00
jerome delattre
76f15ec740 Follow up to 21896012 - change LastPingTime type from float to double
#rnx

[CL 21908291 by jerome delattre in ue5-main branch]
2022-09-08 20:11:58 -04:00
jerome delattre
633b27d9c0 Revert 21824856 (Retry sending a request to run a test if schedule was not confirmed)
#rnx

[CL 21908280 by jerome delattre in ue5-main branch]
2022-09-08 20:11:38 -04:00
jerome delattre
ec6f3a74bc Put back LLM scope and float to double change in test Automation modules
#rnx

[CL 21828531 by jerome delattre in ue5-main branch]
2022-09-06 16:29:09 -04:00
jerome delattre
0998fb6b90 Test Automation - Retry sending a request to run a test if schedule was not confirmed.
#preflight 631767b36431d379928cc011
#rb Chris.Constantinescu
#rnx

[CL 21828390 by jerome delattre in ue5-main branch]
2022-09-06 16:23:26 -04:00
Zak Middleton
84fb0737e0 #ue5 - Fix type conversion warnings in modules: AutomationController, AutomationTest, AVEncoder, AVIWriter.
#jira UE-160830
#rb Andrew.Davidson
#preflight 630d2147556fc14dce527da5

[CL 21690862 by Zak Middleton in ue5-main branch]
2022-08-29 18:32:03 -04:00
Jerome Delattre
2e98fecdba Test Automation - Add Keep PIE open in advance settings
+ add project config settings to change that default.

#jira UE-162059
#preflight 6307d9665a76ca27a8a800d6
#rb Chris.Constantinescu

[CL 21685895 by Jerome Delattre in ue5-main branch]
2022-08-29 15:29:28 -04:00
Jerome Delattre
c6c0e9a2a5 Support individual test exclusion for different RHI when parent suite does not exclude all RHI
+ enhanced management of RHI through the UI: If a RHI is specify, any child test can be excluded on an additional RHI by clicking on edit exclusion
+ disable exclusion editing on remote session

#jira UE-147020
#preflight 62ff9c99086f90bbc401a999
#rb Chris.Constantinescu

[CL 21462884 by Jerome Delattre in ue5-main branch]
2022-08-19 15:29:47 -04:00
bob tellez
410858eb02 #UE Compile fix for when WITH_AUTOMATION_TESTS is false in an editor build
#ROBOMERGE-AUTHOR: bob.tellez
#ROBOMERGE-SOURCE: CL 21342143 via CL 21342255 via CL 21342279
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21343966 by bob tellez in ue5-main branch]
2022-08-11 16:19:05 -04:00
andrew grant
f34aa9015c Fix for automation changes in shipping editor builds
#p4v-cherrypick 21201897

#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21201864 via CL 21204054 via CL 21204077 via CL 21204091
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21207498 by andrew grant in ue5-main branch]
2022-07-21 15:37:21 -04:00
andrew grant
f6c96db43f Automation Improvements for running editor-based automation, and specifically profiling
* Added IgnoreLogEvents automation command / cvar that can be used to exclude log events from automation reports. Specifically here this prevents errors/warnings in something like PIE from manigesting as automation events
* Automation Controller Manager now waits for an interactive framerate before executing. This ensures automation doesn't occur when async loading is being flushedcritical to startup or level loading and f
* Renamed FWaitForAverageFrameRate to FWaitForInteractiveFramerate and moved default values to be ini-driven
* Editor now disables throttling when -benchmarking is specified
* FIxed issue where Project.Maps.PIE only worked if the provided map was a full object name (/Game/Path/MyMap.MyMap)

#rb swarm

#ROBOMERGE-OWNER: andrew.grant
#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21201287 via CL 21201301 via CL 21203967 via CL 21203996 via CL 21204009
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21205507 by andrew grant in ue5-main branch]
2022-07-21 14:39:27 -04:00
bryan sefcik
de1956f47b Ran IWYU on Public headers under Engine/Source/Developer/...
Headers are updated to contain any missing #includes needed to compile and #includes are sorted.  Nothing is removed.

#ushell-cherrypick of 21064294 by bryan.sefcik
#jira
#preflight 62d5c2111062f2e63015e598

#ROBOMERGE-OWNER: bryan.sefcik
#ROBOMERGE-AUTHOR: bryan.sefcik
#ROBOMERGE-SOURCE: CL 21155249 via CL 21158121 via CL 21161259
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21182053 by bryan sefcik in ue5-main branch]
2022-07-20 12:03:45 -04:00
Jerome Delattre
41d280fd6d Fix test number sum of sub category in Test Automation window
+ fix internal test sub category names in Test Automation window

#preflight 62d03045dc4397d38403efb5
#jira none
#rnx
#rb Chris.Constantinescu

[CL 21091332 by Jerome Delattre in ue5-main branch]
2022-07-14 11:21:50 -04:00
Jerome Delattre
164e6417b5 Log Test report path only if it was generated.
#jira none
#rb trivial
#preflight 62cd923b492f42e203039507

[CL 21057294 by Jerome Delattre in ue5-main branch]
2022-07-12 11:49:08 -04:00
chris constantinescu
758a0aa3e8 Automation command line: filter any instead of all when using multiple filters
#rnx
#fyi Sara.Schvartzman
#preflight skip

[CL 21038701 by chris constantinescu in ue5-main branch]
2022-07-11 09:23:30 -04:00
Jerome Delattre
b14815ede7 Tag allocations related to Automation Test components
#jira UE-153882
#rb Ionut.Matasaru, Matt.Peters

#preflight 62c44591c438da7f091bb3b9

[CL 20942097 by Jerome Delattre in ue5-main branch]
2022-07-05 10:45:22 -04:00
chris constantinescu
0d1434c4df Test Groups
#jira UE-127334
#rb Jerome.Delattre
#preflight 62b59f4d0110bdb3313e72e5

[CL 20830416 by chris constantinescu in ue5-main branch]
2022-06-27 10:21:14 -04:00
maxime mercier
0562a84f2e Fixed 2 problems when hitting stop test button,
* the test aren't stop correctly. So the second time you run it, it give you the error that the test is already running.
*  it does not stop PIE even if it started it. But the automation controller manager finishes all the test, then it stops correctly PIE.


#ROBOMERGE-AUTHOR: maxime.mercier
#ROBOMERGE-SOURCE: CL 20519358 via CL 20519368 via CL 20519373
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v954-20466795)

[CL 20521157 by maxime mercier in ue5-main branch]
2022-06-06 11:39:00 -04:00
Jerome Delattre
05eada6e78 Improve errors reported to Automated Test Session
* Support multiple devices on a test (ie: network tests)
* Support log attached to device app instance
* Make Incomplete run flagged as failure when session is finishing
* Critical failure tag as crash in the test report

#preflight 627bffe71e749933439e54ba
#rb Chris.Constantinescu
#jira none

[CL 20378481 by Jerome Delattre in ue5-main branch]
2022-05-26 09:22:47 -04:00
Matt Peters
7ad238a806 AssetRegistry includes (Engine/Source): change #include "AssetData.h" -> #include "AssetRegistry/AssetData.h", and similar for the other moved AssetRegistry headers.
#rb Zousar.Shaker
#rnx
#preflight 6270509a220f89f0ad573030

[CL 20016982 by Matt Peters in ue5-main branch]
2022-05-02 18:06:48 -04:00
Brandon Schaefer
50ca5a9fd3 Move logic into platform request exit area to flush logs if we can
#jira UE-146560
#preflight none
#rb Michael.Sartain, Robert.Seiver

[CL 19440198 by Brandon Schaefer in ue5-main branch]
2022-03-18 15:26:38 -04:00
Brandon Schaefer
5af26f436b GLog->Flush to ensure the UE_LOGs end up in the log, as we require the output of the above logs for a test to pass
#jira UE-146560
#rb Jerome.Delattre
#preflight 6234d2f68900c14eec954a5c

[CL 19439897 by Brandon Schaefer in ue5-main branch]
2022-03-18 15:05:46 -04:00
Jerome Delattre
76a96a81bb Gauntlet - Horde Automated Test Session report
* Introduce Automated Test Session data model for Horde TestData
* Introduce Suite and Type to UnrealTestNode to have better control over Session naming
* Add PreflightChange option to discriminate Session that are not official run
* Add test Duration and DateTime to UE json report model
* Accomodate for posting New and Old report models

#jira UEENGQA-29178
#preflight 622bb8ec46679d56c323f49a
#rb Chris.Constantinescu

[CL 19367745 by Jerome Delattre in ue5-main branch]
2022-03-12 10:03:43 -04:00
Jerome Delattre
e3aa8df0b1 Fix Approved and Unapproved screenshots being mixed up in the json reports
#jira none
#rb trivial
#preflight 6227bde07934051162f351a5

[CL 19310417 by Jerome Delattre in ue5-main branch]
2022-03-08 15:54:17 -05:00
Jerome Delattre
f8c58dfd2c Avoid duplicates of Screenshot comparison artifacts when producing automated test html/json report
#jira UE-144056
#preflight 621e630f3e14f0c7e5497b07
#rb Chris.Constantinescu

[CL 19271563 by Jerome Delattre in ue5-main branch]
2022-03-04 14:04:40 -05:00