128 Commits

Author SHA1 Message Date
Marc Audy
4cab3bc5d7 Fix lost deletes from CL# 35150803
#rnx
#jira

[CL 36772312 by Marc Audy in 5.5 branch]
2024-10-02 01:16:16 -04:00
wojciech krywult
b79f2646ba UnitTests: Unified our unit testing backends to consistently use case insensitive string comparisons. Previously, different backends used different contentions (and were often inconsistent internally) which led to LowLevelTests and EngineTests returning different results for the same tests in a few cases.
- FAutomationTestBase (EngineTests) and LowLevelTestsRunner/Expectations.h use case insensitive comparisons by default.
- Deprecated the existing Test[Not]EqualInsensitive as not it always does the same thing as Test[Not]Equal
- Added Test[Not]EqualSensitive in case we need case sensitive comparisons somewhere.
- Fixed crashes in string comparison overloads for TCHAR*
- Removed superfluous overloads (FString vs TCHAR*) from LowLevelTestsRunner.
- Added TStringView and TUtf8StringView overloads to LowLevelTestsRunner to handle the above and more.
- Added a missing overload for TCHAR* to LowLevelTestsRunner (previously, TCHAR* vs TCHAR* would compare pointer values).
- Added a few additional tests.

#rb Jerome.Delattre, sean.sweeney, sebastian.lewicki
#tests Executed some EngineTests and FoundationTests locally, ran EngineTest and LowLevelTests preflights with my change and compared results to the nightly runs.

[CL 35970884 by wojciech krywult in ue5-main branch]
2024-09-03 12:32:10 -04:00
chris constantinescu
bf4f9cb340 Common assertions header for LLTs and Functional Tests
#jira UE-220179
#rb sean.sweeney
#rnx

[CL 35150803 by chris constantinescu in ue5-main branch]
2024-07-29 16:18:35 -04:00
jonathan bard
1d85c48d81 Added REQUIRE_CHECK_SLOW to low-level test macros, that validates that checkSlow, checkfSlow... are called on configurations where they're effective
#rb Chris.Constantinescu
#tests FoundationTests debug+development

[CL 35051388 by jonathan bard in ue5-main branch]
2024-07-24 08:47:27 -04:00
stephen ma
9be28c4d4f Update LowLevelTests.xml to pass in -programprojectoverride=$(BuildProject) when using StagesWithProjectFile=True
Add support for LowLevelTests uprojects in /Tests/ folder within Game Folders

#rb chris.constantinescu

[CL 34560561 by stephen ma in ue5-main branch]
2024-06-21 09:39:17 -04:00
jimmy smith
231658e788 Fix copy-pasta in AssertionsMacros.h
#rb Devin.Doucette
#rnx
#tests LowLevelTests EnginePreflight

[CL 34210534 by jimmy smith in ue5-main branch]
2024-06-07 14:53:26 -04:00
chris constantinescu
36cd2f25b9 Platform file stub not loading - stub module name changed in CL 33938220 but invoked with previous name
#jira UE-215875

[CL 33983687 by chris constantinescu in ue5-main branch]
2024-05-29 15:33:08 -04:00
martin sevigny
15afa78dab Add test to make sure the module name in the IMPLEMENT_MODULE macros matches the name declared in the uproject, uplugin or the name of the folder for engine modules.
A mismatch for Runtime module results in a error in FModuleManager::LoadModuleWithFailureReason because it can't find the ModuleInitializerPtr

See ModuleManager.h for the actual change. The rest are fixes of wrong names.
#rb Steve.Robb
#jira UE-215245

[CL 33938229 by martin sevigny in ue5-main branch]
2024-05-28 08:46:41 -04:00
chris constantinescu
84bb87d719 Rrestrict test metadata to test mode
Step verified job 66390cb0242b321d4c9d98a6
#jira UE-214088

[CL 33461215 by chris constantinescu in ue5-main branch]
2024-05-06 13:09:50 -04:00
chris constantinescu
a15dd3effe LLT Android missing script that fxes manifest re-added
#jira UE-213725
#rb Chris.Babcock

[CL 33436597 by chris constantinescu in ue5-main branch]
2024-05-03 16:46:50 -04:00
chris constantinescu
5d2d569cb1 Use optional arguments and Extend/Expand pattern for Macros to reduce LLT xml metadata file footprint
.xml metadata files are generated exclusively using TestMode, from values of TestMetadata set from the module class
RunUBT -Mode=Test -GenerateMetadata
To support additional NDA platforms, build modules must be created.
Cleanup unused metadata files from the previous generation system.
#jira UE-195038
#rb Jerome.Delattre

[CL 33373724 by chris constantinescu in ue5-main branch]
2024-05-01 11:35:44 -04:00
chris constantinescu
25d83fecdb Merging
Main/Engine/Build/InstalledEngineFilters.xml
Main/Engine/Source/Developer/LowLevelTestsRunner/LowLevelTestsRunner.Build.cs
Main/Engine/Source/ThirdParty/Catch2/Catch2Extras.Build.cs

to //UE5/Release-5.4/Engine/Source/

#jira UE-210257

[CL 33306012 by chris constantinescu in ue5-main branch]
2024-04-29 13:20:08 -04:00
chris constantinescu
85d4a023e2 Remove Catch2.Version
#jira UE-210257

[CL 33300886 by chris constantinescu in ue5-main branch]
2024-04-29 10:55:56 -04:00
danny couture
75f532dd16 [TaskGraph]
- Remove old task graph backend
 - Remove old deprecations

#rb kevin.macaulayvacher

[CL 33099915 by danny couture in ue5-main branch]
2024-04-19 08:16:10 -04:00
Alex Kasianchuk
8503e37f20 OSS Tests PS4 - Test application didn't log any test events after timeout period of 5 minutes, stopping.
#jira UE-212381
#rb chris.constantinescu, stephen.ma

[CL 33005352 by Alex Kasianchuk in ue5-main branch]
2024-04-16 12:58:38 -04:00
nick edwards
9db1d465da Convert JsonTests to LLT. Add TestFalse to Expectations.h
#tests Ran Json test via FoundationTests locally.
#rb sean.sweeney

[CL 32788868 by nick edwards in ue5-main branch]
2024-04-08 05:53:37 -04:00
chris constantinescu
7c5095a408 LLT Android support
#rb Chris.Babcock, sebastian.lewicki
#jira UE-210332

[CL 32714226 by chris constantinescu in ue5-main branch]
2024-04-03 18:39:56 -04:00
chris constantinescu
56b2fd5e18 Move UE component cleanup logic from TestRunner into proper place (introduced at CL 32031322) and fix refactoring (CL 32031169) on Sony platforms to re-allow unpackaged builds
#jira UE-209196
#rb sebastian.lewicki

[CL 32221092 by chris constantinescu in ue5-main branch]
2024-03-13 14:18:48 -04:00
jamie dale
9bd4d604e4 Tear down localization prior to static destruction
This avoids crashes from a logging attempt during static destruction in the test runner, which can happen as the test runner only manually tears down a subset of things that the standard LaunchEngineLoop handles

#jira UE-208856
[FYI] chris.constantinescu
#rnx

[CL 32031506 by jamie dale in ue5-main branch]
2024-03-05 14:46:10 -05:00
jamie dale
f3631d6d16 Tear down GLog prior to static destruction
This avoids crashes from anything that logs during static destruction in the test runner, which can happen as the test runner only manually tears down a subset of things that the standard LaunchEngineLoop handles

[FYI] chris.constantinescu
#rnx

[CL 31977137 by jamie dale in ue5-main branch]
2024-03-02 11:52:53 -05:00
chris constantinescu
17f311b972 Get active test info implemented into Catch2
#jira UE-201152
#rb rob.huyett, sebastian.lewicki

[CL 30428970 by chris constantinescu in ue5-main branch]
2023-12-21 11:46:29 -05:00
chris constantinescu
e53e2611f7 Catch2 - enable exceptions for all platforms (allows SKIP to work properly)
#rb sebastian.lewicki
#jira UE-201146

[CL 30216424 by chris constantinescu in ue5-main branch]
2023-12-08 14:07:38 -05:00
chris constantinescu
0f67165952 Fix compilation error when building implcit tests
#rb sebastian.lewicki, stephen.ma
#jira UE-201512

[CL 29999694 by chris constantinescu in ue5-main branch]
2023-11-29 16:58:22 -05:00
chris constantinescu
6960d95832 Upgrade Catch2 to 3.4.0
#jira UE-198978
#rb sebastian.lewicki-qa-sdets

[CL 29897708 by chris constantinescu in ue5-main branch]
2023-11-22 16:52:08 -05:00
rafa lecina
92ff150bd2 Improve logs for LowLeveltests by including captured/scoped info data when assertions are triggered
#jira UE-192858
[REVIEW] [at]Stephen.Ma, [at]Chris.Constantinescu, [at]Michael.Kirzinger

[CL 28378083 by rafa lecina in ue5-main branch]
2023-10-02 10:51:26 -04:00