More specifically, all low level tests were deployed into LowLevelTests directory/sandbox. It meanst that if we ran OnlineTests, which deploy .ini scripts, a subsequent run of FoundationTests would have these .ini as well, which would unintentionally affect the behavior of FoundationTests.
Now, Online will be deployed into LowLevelTests-OnlineTests and Foundation into LowLevelTests-Foundation avoiding the clash.
#rb chris.constantinescu, Jerome.Delattre
#jira UE-196287
#rnx
[CL 28548968 by wojciech krywult in ue5-main branch]
Redirected binaries so they end up on the proper path
#jira UE-192858
[REVIEW] [at]Michael.Kirzinger, [at]Stephen.Ma, [at]Chris.Constantinescu, [at]Josh.Adams
[CL 27829673 by rafa lecina in ue5-main branch]
- Catch2 built for LinuxArm64
- new option to run apps in containerized mode
- builds docker image, setup and cleanup
- StringOp BuildGraph element for string operations (replace, to lower etc)
#jira UE-193565
#rb ue-qa-sdets
[CL 27758925 by chris constantinescu in ue5-main branch]
- LowLevelTests.xml Builds tests under BuildCookRun with -NoBootstrapExe
- TestModuleRules no longer generates arguments -BaseFromWorkingDir to generated .xml files
- Fixes bug in RunLowLevelTests that puts tags in the wrong position in command line arguments
#jira UE-188819
[CL 26994753 by stephen ma in ue5-main branch]
- LowLevelTest.xml horde build step as a separate label
- Relevant <General>.xml files updated to support the new build step
- Added UpdateBuildGraphPropertiesFile.Automatoin.cs to help find files to checkout
- OnlineTests updated to utilize new BuildCookRun param
#jira UE-188819
#rb Chris.Constantinescu, Jake.Niman
[CL 26195017 by stephen ma in ue5-main branch]
#rb Chris.Constantinescu, Michael.Kirzinger, Jake.Niman
#fyi Alejandro.Aguilar, Chris.Varnsverry, Lorri.Li
#jira UE-184491
#preflight 644a7c2009453df5485e95be
[CL 25214478 by stephen ma in ue5-main branch]
Default timeouts for tests are:
- optional, only enabled when set via --timeout=X (in minutes)
- standard practice to help triage tests and bound their running time; slower tests can be tagged appropriately and run with the nightly tests
- useful to detect segmentation faults and other issues by enforcing a maximum period of inactivity
#rb Jerome.Delattre, Adam.Miezianko, Rob.Huyett
#jira UE-182253
#preflight 642f18864c3ccbbdf18b9916
[CL 25067632 by chris constantinescu in ue5-main branch]
Additional small quality of life fixes
#rb Jerome.Delattre, Rob.Huyett
#preflight 64120656bd6894c435c1f7c4
[CL 24660566 by chris constantinescu in ue5-main branch]
- remove dependency on LaunchEngineLoop.cpp which also allows compilation against engine to succeed
- LowLevelTestsRunner no dependency on engine modules required: Launch, Core, Project
- EXPLICIT_TESTS_TARGET definition for self-contained tests: test modules and targets that derive from TestModuleRules/TestTargetRules respectively
- Cleanup Launch dependencies from existing explicit tests
- Cleanup redundant flags from existing explicit tests
- rename Self -> Foundation for in Horde
- additional platform fixes
Default #preflight 63e14d37244dc45a20e29337
All platform/tests LLTs #preflight 63e022f91b44ee7cb1c11d60
#rnx
[CL 24035900 by chris constantinescu in ue5-main branch]
Have gauntlet display the command a test was ran with
Add server output to the horde log so errors can be easier identified from a webpage.
Fixing a typo with Text which should have been Test
#preflight 63408864587d6afac86da474
#rb: Gary.Yuan
https://p4-swarm.epicgames.net/reviews/22408413/
[CL 22451857 by eric knapik in ue5-main branch]
- bump "Started test case" log messages to Display when --debug is used
- additional duration log messages. Catch2 durations were never printed in sync with --debug log messages
- ability to specify a device and attach to a debugger when running the LowLevelTests script; ability to specify configuration type
#rb Devin.Doucette, Jerome.Dellatre
#preflight 631f554d705bdbceb5b91b82
[CL 21989658 by chris constantinescu in ue5-main branch]
- sleep on init to allow for sync w/ Gauntlet for Xbox platforms
- remove unused --base-global-module #fyi Stephen.Ma
Gauntlet:
- support low level tests build for different configurations (debug etc)
- readable command line arguments (-r to --reporter)
- allow argument names containing "-" e.g. --filenames-as-tags
#preflight 630536b1c00af5e2943540a1
#rb Jerome.Delattre
[CL 21550045 by chris constantinescu in ue5-main branch]
AssertionMacrosTest is printing out failed asserts using LocalPrint which is undesirable - implemented GBlockLocalOutput as a way to supress LocalPrint output.
Other fixes included:
- ability to select a low level test build based on any configuration, not just Development: useful for Debug builds
- New "printoutput" LLT Gauntlet parameter used for certain platforms to control output redirect to stdout
#rb David.Harvey
#preflight 62f405a05a6f2fec99b64f21
[CL 21334255 by chris constantinescu in ue5-main branch]
- add virtual devices to the pool, for parity with RunUnreal.
- release devices when the session is destroyed.
#jira UE-139998
#rb Chris.Constantinescu
#preflight 62a1a81644115ef87db55997
[CL 20572593 by David Harvey in ue5-main branch]
