Move enum from AutomationUtils.Automation to BuildUtilities
#jira none
#trivial
#preflight 60ca8a3d78c3b00001f86b24
[CL 16701095 by jonathan adamczewski in ue5-main branch]
Improved behavior of PIE automation to deal with extremly long hitches and better represet the time a user is going to wait before the editor is usable.
Improved usability of BenchmarkBuild script
Changes
+ Added new automation command for waiting until average fps reaches a specified value. Using this with a conservative value is a better way of checking for interactivity than looping at shader compiles etc.
+ When Errors are added to a test as a result of logging, this itself is now logged as an error. This makes it clearer why a test may have failed despite its own checks having passed.
+ BenchmarkBuild can now just specify empty arguments for PIE/Cook tests (e.g. -PIE1_Args="") to allow comparisons between default/other.
+ Project.Maps.PIE can now take a "-maps" argument on the command line for the list of maps to test
+ Turned on AutomationController.SuppresLogErrors for Frosty to prevent errors causing errors to fail
+ Cleaned up naming for mulltiple-variations of PIE/Cook tests in BenchmarkBuild. Now use -PIE1Args, -PIE2Args etc
Fixes -
* Fixed issue in AutomationControllerManager where extreme hitching resulted in an editor test timing out due to outstanding pings being given a "TimeSince" that would exceed a timeout
* Fixed issue where bTreatLogErrorsAsTestErrors was (strangely) causing the editor to exit with a zero-error code when tests failed.
* Made FWaitForShadersToFinishCompiling automation task non-blocking to prevent timeouts. (It also checks texture compilation as the blocking version used to)
* Fixed BenchmarkBuild using the old System.Maps.PIE test that isn't enabled by default in projects
* Removed flush of shaders/textures in Project.Maps.PIE since this doesn't represent when the level is playable. Measuring this will be added as a separate test.
* Fixed issue where a user specifying a map with an extension would cause Project.Maps.PIE not to recognize the level had loaded
#rb swarm
#jira nojira
[CL 15426493 by Andrew Grant in ue5-main branch]
- made noxge apply to shader compiles
- added option to test backends
- tweaked some cleanup stuff.
#ROBOMERGE-SOURCE: CL 12486820 via CL 12486833 via CL 12486836 via CL 12486838
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v673-12478461)
[CL 12487602 by andrew grant in Main branch]
used safer means of deleting old directories
#ROBOMERGE-SOURCE: CL 12216928 via CL 12216959 via CL 12222525
#ROBOMERGE-BOT: (v663-12210973)
[CL 12222689 by andrew grant in Main branch]
Added option to select map to PIE to
Fixed issue with clients being compiled when not specified
#ROBOMERGE-SOURCE: CL 12127450 via CL 12127660 via CL 12127677 via CL 12132848
#ROBOMERGE-BOT: (v659-12123632)
[CL 12133065 by andrew grant in Main branch]
[FYI] josh.engbretson
#ROBOMERGE-SOURCE: CL 12094798 via CL 12094804 via CL 12094805 via CL 12094808
#ROBOMERGE-BOT: (v657-12064184)
[CL 12094810 by andrew grant in Main branch]
Added NoDDC option for PIE tests.
ColdDDC option no longer nukes the local DDC, instead a temporary path is used.
Reworked PIE / Cook tests to rely more on a common base class functionality.
#rb swarm
#tests ran BenchmarkBuild with different options
#ROBOMERGE-SOURCE: CL 12056559 via CL 12056560 via CL 12056561 via CL 12056562
#ROBOMERGE-BOT: (v656-11643781)
[CL 12056563 by andrew grant in Main branch]
#ROBOMERGE-SOURCE: CL 12000189 via CL 12000732 via CL 12002271 via CL 12005776
#ROBOMERGE-BOT: (v656-11643781)
[CL 12048174 by andrew grant in Main branch]
Added EditorStartup test
#ROBOMERGE-SOURCE: CL 11943314 via CL 11944263 via CL 11944396 via CL 11944577
#ROBOMERGE-BOT: (v656-11643781)
[CL 11947228 by andrew grant in Main branch]
