Commit Graph

67 Commits

Author SHA1 Message Date
sean sweeney
94cb8cf0ce Test tag filtering via CLI and CVar using ExecCmd, exposes tag filtering to existing INI groups.
Exposes tag registration from CL 34851931 to users who use the UE Editor console or CLI args. Does not yet plumb tags through non-C++ interfaces, such as Gauntlet or Blueprints. Also does not connect tags to the Automation pane of the Session Frontend UI.

#jira UE-219492
#tests Ran existing tagged tests in editor by executing commands in the console such as: Automation SetTagFilter [TestExampleTag] ... Automation RunTest StartsWith:TestFramework
#rb jerome.delattre

[CL 35509105 by sean sweeney in ue5-main branch]
2024-08-13 17:11:22 -04:00
steve robb
85f2d450ad EAutomationTestFlags is now an enum class of almost entirely pure flags, with the aim of making the flags visible in the debugger. ApplicationContextMask still needs fixing up everywhere.
#rb robert.manuszewski

[CL 34305488 by steve robb in ue5-main branch]
2024-06-12 09:14:05 -04:00
jerome delattre
60c82b6e18 Align commandline filter options to test automation window filter options.
Add Standard and Negative option

#rnx

#rb sebastian.lewicki

[CL 33733779 by jerome delattre in ue5-main branch]
2024-05-17 15:26:22 -04:00
jerome delattre
25d929e456 Fix an issue with Automation Command line not being initiated properly
#rnx

[CL 30343851 by jerome delattre in ue5-main branch]
2023-12-15 02:13:35 -05:00
jerome delattre
52ad19c319 Fix issue with Automation commands conflicting with Automation CVar
#rnx
#rb rob.huyett, sebastian.lewicki

[CL 30302130 by jerome delattre in ue5-main branch]
2023-12-13 15:18:55 -05:00
jerome delattre
e898ae373b Avoid running all tests if Group filter does not exist
#rnx

#rb chris.constantinescu, sebastian.lewicki

[CL 30184386 by jerome delattre in ue5-main branch]
2023-12-07 12:51:00 -05:00
christopher fiala
a0d65ab7f0 Move the global stereo test variants CVar out of ScreenshotFunctionalTest.cpp and into AutomationTest.cpp as "Automation.EnableStereoTestVariants", where it can now be controlled via the automation commandline
(e.g. "Automation Runtest Anisotropy;EnableStereoTests;Quit;").

#jira UE-194361
#rb jeff.fisher

[CL 29529430 by christopher fiala in ue5-main branch]
2023-11-07 14:57:30 -05:00
jerome delattre
25cc6a7006 Fix infinite loop when Quit command is input more than once
#rnx
#rb Clayton.Langford

[CL 26150778 by jerome delattre in ue5-main branch]
2023-06-21 11:26:41 -04:00
Jerome Delattre
86452037e1 Fix ExecCmds Automation queueing command line to support input like -exccmds="Automation RunTest MyTest,Automation List,Automation Quit"
#rnx
#rb Chris.Constantinescu

[CL 26050022 by Jerome Delattre in ue5-main branch]
2023-06-16 09:52:35 -04:00
jodon karlik
80e2303e74 Adding a GameplayTagQuery field to FGameplayTagRequirements to enable more complex requirements to be specified.
[CL 25833608 by jodon karlik in ue5-main branch]
2023-06-06 20:26:28 -04:00
galeone
431c09150e [UBT] Add support for Code coverage on Linux
#rb Brandon.Schaefer, Jerome.Delattre, Joe.Kirchoff

[CL 25794147 by galeone in ue5-main branch]
2023-06-05 12:03:25 -04:00
kirill zorin
b6ee3a6c64 Fix UE_LOG callsites that have format string-related UB
#rb trivial
#preflight 647107780a6634dbb236a0a6

[CL 25651638 by kirill zorin in ue5-main branch]
2023-05-26 15:52:39 -04:00
jeanmichel dignard
8d07a7338f Exec commands changes:
- Port some development only exec commands to Exec_Dev.

#rb none
#preflight 63c5c97f02024f93d822d678

[CL 23734254 by jeanmichel dignard in ue5-main branch]
2023-01-16 19:28:53 -05:00
Jerome Delattre
a780c91ede AutomationController - Fixing end queue message not being triggered
Prevent multiple Run Test to be queued at the same time (not supported)
Rename Filter: to StartsWith:
Some more code cleanup

#jira UE-170942
#rnx
#rb Ben.Salem
#preflight 638f68315c5308d18ce3c7ee

[CL 23416127 by Jerome Delattre in ue5-main branch]
2022-12-06 15:49:46 -05:00
timrademaker
560e9cecd9 PR #7401: Fix/AutomationCommandLine (Contributed by timrademaker)
#rb Jerome.Delattre
#preflight skip

[CL 22013755 by timrademaker in ue5-main branch]
2022-09-14 15:21:59 -04:00
andrew grant
f6c96db43f Automation Improvements for running editor-based automation, and specifically profiling
* Added IgnoreLogEvents automation command / cvar that can be used to exclude log events from automation reports. Specifically here this prevents errors/warnings in something like PIE from manigesting as automation events
* Automation Controller Manager now waits for an interactive framerate before executing. This ensures automation doesn't occur when async loading is being flushedcritical to startup or level loading and f
* Renamed FWaitForAverageFrameRate to FWaitForInteractiveFramerate and moved default values to be ini-driven
* Editor now disables throttling when -benchmarking is specified
* FIxed issue where Project.Maps.PIE only worked if the provided map was a full object name (/Game/Path/MyMap.MyMap)

#rb swarm

#ROBOMERGE-OWNER: andrew.grant
#ROBOMERGE-AUTHOR: andrew.grant
#ROBOMERGE-SOURCE: CL 21201287 via CL 21201301 via CL 21203967 via CL 21203996 via CL 21204009
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21205507 by andrew grant in ue5-main branch]
2022-07-21 14:39:27 -04:00
chris constantinescu
758a0aa3e8 Automation command line: filter any instead of all when using multiple filters
#rnx
#fyi Sara.Schvartzman
#preflight skip

[CL 21038701 by chris constantinescu in ue5-main branch]
2022-07-11 09:23:30 -04:00
chris constantinescu
0d1434c4df Test Groups
#jira UE-127334
#rb Jerome.Delattre
#preflight 62b59f4d0110bdb3313e72e5

[CL 20830416 by chris constantinescu in ue5-main branch]
2022-06-27 10:21:14 -04:00
Matt Peters
7ad238a806 AssetRegistry includes (Engine/Source): change #include "AssetData.h" -> #include "AssetRegistry/AssetData.h", and similar for the other moved AssetRegistry headers.
#rb Zousar.Shaker
#rnx
#preflight 6270509a220f89f0ad573030

[CL 20016982 by Matt Peters in ue5-main branch]
2022-05-02 18:06:48 -04:00
Brandon Schaefer
50ca5a9fd3 Move logic into platform request exit area to flush logs if we can
#jira UE-146560
#preflight none
#rb Michael.Sartain, Robert.Seiver

[CL 19440198 by Brandon Schaefer in ue5-main branch]
2022-03-18 15:26:38 -04:00
Brandon Schaefer
5af26f436b GLog->Flush to ensure the UE_LOGs end up in the log, as we require the output of the above logs for a test to pass
#jira UE-146560
#rb Jerome.Delattre
#preflight 6234d2f68900c14eec954a5c

[CL 19439897 by Brandon Schaefer in ue5-main branch]
2022-03-18 15:05:46 -04:00
phisko
6879eb72a5 PR #8106: Add SetFilter command to Automation framework (Contributed by phisko)
#jira UE-117305
#rb Eric.Knapik

[CL 17650998 by phisko in ue5-main branch]
2021-09-28 16:11:26 -04:00
Andriy Tylychko
2a295eb685 deprecated FTicker and family and replaced by thread-safe FTSTicker
#jira UE-120090
#rb francis.hurteau

[CL 17176325 by Andriy Tylychko in ue5-main branch]
2021-08-16 11:05:18 -04:00
aurel cordonnier
d17d20ca36 Merge from Release-Engine-Test @ 16758890 to UE5/Main
This represents UE4/Main @ 16738161 and Dev-PerfTest @ 16737719 (and Release-17.00 @ 16658211)

[CL 16763350 by aurel cordonnier in ue5-main branch]
2021-06-23 17:51:32 -04:00
aurel cordonnier
43fa62fcd8 Merge from Release-Engine-Test @ 16487383 to UE5/Main
This represents UE4/Main @ 16445039 and Dev-PerfTest @ 16444526

[CL 16488106 by aurel cordonnier in ue5-main branch]
2021-05-27 13:40:37 -04:00