- test-specific targets and modules are constructed by RulesAssembly which removes a lot of spaghetti code
- no more excluding the Tests folder by default which is guaranteed to mislead users #fyi Jon.Nabozny
- test compilation in Tests is controlled through WITH_LOW_LEVEL_TESTS
- preparation code for switching tests between Catch2 / TestAutomation Fmwk
Nightly LLT PF 6260431391376845adeb4c40
#rnx
#preflight 62602d50dd47b4ad2173c30b
[CL 19834665 by chris constantinescu in ue5-main branch]
Motivation: CoreUObject, taskgraph and threads are global constructs and need to be initialized as such and are expensive to setup/teardown before/after multiple single tests.
Introduced --global-setup and --no-global-setup to control execution of global test setup.
Unit tests that rely heavily on mocking and are designed to run without any global context should use --no-global-setup
Other: Removed CoreTestFixture and applied minor fixes wherever things didn't make sense.
One particular CoreUObject test was converted but had always failed to run on non-desktop platforms. It needs to be addressed ASAP.
#preflight 625873c7e304a95465fc148e
#rb Devin.Doucette, Zousar.Shaker
[CL 19769262 by chris constantinescu in ue5-main branch]
Prior to this change running a catch executable (e.g., I tested with GeometryProcessingUnitTests.exe) there was no way to see any UE_LOGs or ensure output. Following this change ensure output is always shown, UE_LOG output with verbosity level <= Warning is shown if the command line argument "--log" is passed, and UE_LOG output with verbosity level <= Log is shown if the command line arguement "--debug" is also passed. For even more verbose output the `DesiredLogVerbosity` variable in TestRunner.cpp can be edited.
#rb jimmy.andrews
#rnx
#jira none
#preflight 6255dc50153828d2734dbe7c
[CL 19728147 by matija kecman in ue5-main branch]
Correcting tests commited in CL 19498448 that were converted and tagged [Smoke] but weren't marked with EAUtomationTestFlags::SmokeFilter in the original source.
EditorBulkDataTests was left unconverted, moved back to original Private/Tests/Serialization folder
ObjectPtrTest -> CoreUObject::TObjectPtr::Soft Object Path fails at line 197 on non-desktop platforms but it didn't failed before the conversion.
#preflight 624eee0a2c01f9c6da9446e0
#rb Devin.Doucette
[CL 19668909 by chris constantinescu in ue5-main branch]
Motivation: [.] only hides a test and it can still be executed using ~[SomeOtherTag]; It's very easy to fall into the trap of executing an infinite loop test that is decorated with the [.] tag
#rb Andriy.Tylychko, Michael.Balzer
#preflight 6245fb64f4217035fa8f38ec
[CL 19576763 by chris constantinescu in ue5-main branch]
Not all were converted because they were designed for packaged applications whereas low level tests are designed to run as "native" non-packaged applications.
- reporting support for non-desktop platforms, for the moment Catch2 report type "console" is used that is sent to a .out file
- most number of tests possible running multi platform, slower tests excluded on incremental builds
- slow tests are moved to run on the monolithic build
- Catch2 report failure event listener such that Horde detects them as build errors. Must add new Horde matcher for Catch2 failures
#rb Mark.Lintott
#preflight 623c8de389625f0612dabd64
[CL 19498448 by chris constantinescu in ue5-main branch]
- Adds method to specify Global Initalization of LowLevelTest Modules
- Fixes math header issue in LowLevelTestModule on Mac
#preflight 62335d5a1302f69e9a483a59
#rb @chris.constantinescu
[CL 19423270 by stephen ma in ue5-main branch]
Editor, Engine, CoreUObject etc will have their own separate utility headers for tests that rely on respective special modules.
Replace platform names in catch2 code.
LLT all tests run preflight: 621e3c91f1206ae3ea5b0a66
#preflight 621e3c8bf1206ae3ea5b0872
#rb Jerome.Delattre
[CL 19202751 by chris constantinescu in ue5-main branch]
- This should avoid the startup.cpp and teardown.cpp pattern and more predictiablly tear-down and setup tests during GlobalSetup/Teardown.
#jira OI-3788
#review @chris.constantinescu
#rb @chris.constantinescu
#preflight 6217e996785f20248baf285d
[CL 19124450 by stephen ma in ue5-main branch]
- optional alter of source file build order: towards beginning or end
- test build: correctly handle inclusion of special dependencies such as Engine, Editor, CoreUObject and ApplicationCore
- existing tests: add setup and teardown tests that initialize test modules. This causes setup and teardown to be local to module instead of global, which was not the correct way to run tests across multiple modules.
#preflight 62151e43dc0170cfb3649ee8
#rb Jermoe.Delattre
[CL 19076600 by chris constantinescu in ue5-main branch]
- Addressed remaining NDA platform code that was present in public facing folders
- Compile LowLevelTests target by default on presubmits and incremental builds for Main and 5.0
- Add dummy test on LowLevelTetsts and run it on consoles daily - this test is called "Self" and it's a sanity check run for Catch2
- Fixed Switch indefinite hang - Self test run successfully on this console
- Added IRunningStateOptions to control app run state: startup and check running state options
- AudioUnitTests run successfully on XboxOneGDK and XSX
- XSX Self and AudioUnitTests run successfully but XSX reports VideoEscape errors - JIRA UE-131334
#jira UEENGQA-52681, UE-127449
#rb Jerome.Delattre
#robomerge 5.0
[CL 17830364 by chris constantinescu in ue5-main branch]
- fix for async loading thread assert on platforms that require cooked data.
#jira UE-127389
#robomerge 5.0
#rb Eric.McDaniel, Chris.Constantinescu
#rnx
[CL 17746872 by David Harvey in ue5-main branch]
Intended for writing unit, integration, functional and all types of tests.
#jira UEENGQA-49764
#rb Jerome.Delattre
[CL 17666358 by chris constantinescu in ue5-main branch]
