48 Commits

Author SHA1 Message Date
chris constantinescu
56b2fd5e18 Move UE component cleanup logic from TestRunner into proper place (introduced at CL 32031322) and fix refactoring (CL 32031169) on Sony platforms to re-allow unpackaged builds
#jira UE-209196
#rb sebastian.lewicki

[CL 32221092 by chris constantinescu in ue5-main branch]
2024-03-13 14:18:48 -04:00
jamie dale
9bd4d604e4 Tear down localization prior to static destruction
This avoids crashes from a logging attempt during static destruction in the test runner, which can happen as the test runner only manually tears down a subset of things that the standard LaunchEngineLoop handles

#jira UE-208856
[FYI] chris.constantinescu
#rnx

[CL 32031506 by jamie dale in ue5-main branch]
2024-03-05 14:46:10 -05:00
jamie dale
f3631d6d16 Tear down GLog prior to static destruction
This avoids crashes from anything that logs during static destruction in the test runner, which can happen as the test runner only manually tears down a subset of things that the standard LaunchEngineLoop handles

[FYI] chris.constantinescu
#rnx

[CL 31977137 by jamie dale in ue5-main branch]
2024-03-02 11:52:53 -05:00
chris constantinescu
28361ac31d LLT: Fix no stdout for Linux tests
#jira UE-195640
#rb ue-qa-sdets

[CL 28199971 by chris constantinescu in ue5-main branch]
2023-09-25 15:21:03 -04:00
dave haslam
915b1b9c01 [VOS] add FOptionProperty serialization support and tests (1/2)
- copies changes for LowLevelTestRunner to 26.00

[at]chris.constantinescu

[CL 26813816 by dave haslam in ue5-main branch]
2023-08-03 12:09:49 -04:00
chris constantinescu
c45d474014 UnrealVS - do not project prepend project path for Editor configuration in the case of test targets
- Project generation will add a new IsTestTarget property in the "Globals" property group section of the project file for explicit test target projects
- UnrealVS checks for this property and does not change the command line for test targets
- TestRunner removes all project argument logic - that can be passed after --extra-args using -project="Path/To/*.uproject" and/or -projectdir="Path/To/Project"
- OSS and Online global handlers, which occur during the Catch2 session, will read project name and/or directory using SetProjectNameAndDirectory from TestCommon/Initialization.h
#jira UE-189840

[CL 26688348 by chris constantinescu in ue5-main branch]
2023-07-28 17:20:05 -04:00
stephen ma
cee64d03a5 LowLevelTests: Fix executable name being used as project path instead of the .uprogram when present.
[CL 26576133 by stephen ma in ue5-main branch]
2023-07-25 11:48:15 -04:00
stephen ma
68d83932f5 LowLevelTestRunner: Fix bug when both .uproject and --projectdir is present, projectdir is incorrectly parsed as a .uproject path instead of a directory path.
#jira UE-190516

[CL 26438719 by stephen ma in ue5-main branch]
2023-07-18 13:09:59 -04:00
chris constantinescu
6072c762ad Minor: TestRunner.cpp --no-wait option fix, generate metadata config must be in target file not build file.
Enable in BaseEngie.ini
[LowLevelTestsSettings]
bUpdateBuildGraphPropertiesFile=false

#rb Jerome.Delattre, Rob.Huyett
#jira UE-189825

[CL 26323717 by chris constantinescu in ue5-main branch]
2023-06-29 15:18:42 -04:00
stephen ma
c94bfd0067 LowLevelTests: Fix bug where --projectdir= was only allowed as the first argument. Its now allowed anywhere on path.
[CL 26210283 by stephen ma in ue5-main branch]
2023-06-23 11:02:30 -04:00
stephen ma
3dbd591598 LowLevelTests: Fix issue with nested && || warning with GCC.
[CL 26198401 by stephen ma in ue5-main branch]
2023-06-22 19:10:53 -04:00
stephen ma
d1087a6cb5 LowLevelTests: Add ability to set -ProjectDir via first paramter ending in .uproject or via -projectdir.
This should allow running LowLevelTests correctly via UnrealVS.

#jira UE-188819

[CL 26194548 by stephen ma in ue5-main branch]
2023-06-22 17:27:45 -04:00
robert millar
e18844e14d Add -waitfordebugger argument matching to low level test runner to match editor/game arguments
#rb none

[CL 25986029 by robert millar in ue5-main branch]
2023-06-14 13:42:52 -04:00
joe pribele
424b17c8c1 always initialize GWarn as its used by FScopedSlowTask
https://p4-swarm.epicgames.net/reviews/25633852

#rb Chris.Constantinescu
#preflight 6470d8669c600ef1537ddb82

[CL 25650674 by joe pribele in ue5-main branch]
2023-05-26 15:07:10 -04:00
chris constantinescu
a1df65eeb5 Low Level Tests: timeout detection from within test app, inactivity detection via Gauntlet
Default timeouts for tests are:
- optional, only enabled when set via --timeout=X (in minutes)
- standard practice to help triage tests and bound their running time; slower tests can be tagged appropriately and run with the nightly tests
- useful to detect segmentation faults and other issues by enforcing a maximum period of inactivity
#rb Jerome.Delattre, Adam.Miezianko, Rob.Huyett
#jira UE-182253
#preflight 642f18864c3ccbbdf18b9916

[CL 25067632 by chris constantinescu in ue5-main branch]
2023-04-17 10:11:51 -04:00
Matt Peters
b76ee66366 Add -buildmachine to LowLevelTests, so that DDC performance warnings can be downgraded to display.
#rnx
#rb Zousar.Shaker, Devin.Doucette
#preflight 63fd16fe437ce3e7f3449cce

[CL 24434214 by Matt Peters in ue5-main branch]
2023-02-27 17:44:59 -05:00
chris constantinescu
5a3a0cc990 Online LLT fix global setup
#rb stephen.ma
#preflight 63ee6fed4d2bd4346664ce25

[CL 24265150 by chris constantinescu in ue5-main branch]
2023-02-16 14:19:09 -05:00
chris constantinescu
4e7e7f9f3f LLT cleanup and improvements/updates:
- remove dependency on LaunchEngineLoop.cpp which also allows compilation against engine to succeed
- LowLevelTestsRunner no dependency on engine modules required: Launch, Core, Project
- EXPLICIT_TESTS_TARGET definition for self-contained tests: test modules and targets that derive from TestModuleRules/TestTargetRules respectively
- Cleanup Launch dependencies from existing explicit tests
- Cleanup redundant flags from existing explicit tests
- rename Self -> Foundation for in Horde
- additional platform fixes
Default #preflight 63e14d37244dc45a20e29337
All platform/tests LLTs #preflight 63e022f91b44ee7cb1c11d60
#rnx

[CL 24035900 by chris constantinescu in ue5-main branch]
2023-02-06 14:27:24 -05:00
chris constantinescu
d1ad7b190a Online/ OSS tests: Catch register tests using delegates, setup delegate is invoked during TestRunner::GlobalSetup and teardown during TestRunner::GlobalTeardown
#preflight 637501d9b663683828111981
#rb Michael.Kirzinger

[CL 23155638 by chris constantinescu in ue5-main branch]
2022-11-16 11:50:06 -05:00
chris constantinescu
7859a6cdb4 Catch2 feature introduced: test groups and group events at global, group and test group before/after levels
#preflight 636ab505450be1d9f83e90e4
#rb Jerome.Delattre

[CL 23039159 by chris constantinescu in ue5-main branch]
2022-11-08 16:02:46 -05:00
joe pribele
7938bdda07 [LowLevelTestRunner] added custom reporter to make parsing of xml easier for UnrealVS TestAdapter
#rb devin.doucette
#preflight 635312f3777a77c4407b2733

[CL 22702551 by joe pribele in ue5-main branch]
2022-10-21 18:00:09 -04:00
joe pribele
1366bc3f93 [Core] added support to lowlevel tests validating that ensure and check are executed
added  a tls for an ensure handler callback to let the reporting log of ensures to be overridden

the following macros are supported in lowlevel tests
REQUIRE_ENSURE
CHECK_ENSURE
REQUIRE_CHECK

#rb devin.doucette
#preflight 634827c457048bdb82b43ffe

[CL 22505260 by joe pribele in ue5-main branch]
2022-10-13 11:24:15 -04:00
jake niman
d8831901d1 Fix a crash on shutdown with LowLevelTests- some threads still may be accessing FCommandLine right as we're tearing down so make sure it is set properly.
#jira UE-163284
#review-22032414 @stephen.ma @chris.constantinescu
#preflight 63236351c225e8be380d130d

[CL 22033267 by jake niman in ue5-main branch]
2022-09-15 14:01:41 -04:00
chris constantinescu
08fee9ccbf Low level tests quality of life changes:
- bump "Started test case" log messages to Display when --debug is used
- additional duration log messages. Catch2 durations were never printed in sync with --debug log messages
- ability to specify a device and attach to a debugger when running the LowLevelTests script; ability to specify configuration type
#rb Devin.Doucette, Jerome.Dellatre
#preflight 631f554d705bdbceb5b91b82

[CL 21989658 by chris constantinescu in ue5-main branch]
2022-09-13 15:06:34 -04:00
devin doucette
6a0aa068b2 LowLevelTests: Restored previous indentation in TestRunner.cpp
#jira none
#preflight skip
#rb none
#rnx

[CL 21975739 by devin doucette in ue5-main branch]
2022-09-12 19:26:01 -04:00