Commit Graph

46 Commits

Author SHA1 Message Date
jake niman
d8831901d1 Fix a crash on shutdown with LowLevelTests- some threads still may be accessing FCommandLine right as we're tearing down so make sure it is set properly.
#jira UE-163284
#review-22032414 @stephen.ma @chris.constantinescu
#preflight 63236351c225e8be380d130d

[CL 22033267 by jake niman in ue5-main branch]
2022-09-15 14:01:41 -04:00
chris constantinescu
b2e0214c64 (LLT) Set cached data directories via FPlatformFile::InitializeAfterProjectFilePath -> fixes code path cycle for Program type applications where no project path is defined.
#jira UE-163672
#rb Josh.Adams
#preflight 6321e55e506f1a33e0bf2576

[CL 22010754 by chris constantinescu in ue5-main branch]
2022-09-14 13:37:32 -04:00
chris constantinescu
08fee9ccbf Low level tests quality of life changes:
- bump "Started test case" log messages to Display when --debug is used
- additional duration log messages. Catch2 durations were never printed in sync with --debug log messages
- ability to specify a device and attach to a debugger when running the LowLevelTests script; ability to specify configuration type
#rb Devin.Doucette, Jerome.Dellatre
#preflight 631f554d705bdbceb5b91b82

[CL 21989658 by chris constantinescu in ue5-main branch]
2022-09-13 15:06:34 -04:00
devin doucette
6a0aa068b2 LowLevelTests: Restored previous indentation in TestRunner.cpp
#jira none
#preflight skip
#rb none
#rnx

[CL 21975739 by devin doucette in ue5-main branch]
2022-09-12 19:26:01 -04:00
yiliang siew
ab43620f1b Update path to Catch2 sources.
#rb Chris.Constantinescu
#preflight 631a109c304480f8f85a8c46

[CL 21916057 by yiliang siew in ue5-main branch]
2022-09-09 00:55:28 -04:00
Devin Doucette
8d4b17d3d2 LowLevelTests: Always shut down modules on exit
It is unsafe to skip module shutdown because many modules need to destroy things that were lazily initialized, and may crash later during shutdown if the expected shutdown flow was skipped.

#jira UE-163043
#preflight 631a272a980179553ed8097d
#preflight 631a27eea60c539c989bc821
#rb Brandon.Schaefer
#rnx
#fyi Chris.Constantinescu

[CL 21902718 by Devin Doucette in ue5-main branch]
2022-09-08 16:35:00 -04:00
Devin Doucette
4375283ed6 LowLevelTests: Removed an unnecessary #include
#preflight 6317a873967ffc68fb6e7259
#rb none
#rnx

[CL 21850805 by Devin Doucette in ue5-main branch]
2022-09-07 10:38:37 -04:00
chris constantinescu
714b6470c8 LLT TestRunner non-STL argument value parsing
#rb Devin.Doucette
#preflight 630fb502556fc14dce9df430

[CL 21739890 by chris constantinescu in ue5-main branch]
2022-09-01 10:12:37 -04:00
chris constantinescu
b192ea74b9 Sleep uses seconds, not milliseconds
#rnx
#preflight skip

[CL 21562824 by chris constantinescu in ue5-main branch]
2022-08-25 09:30:44 -04:00
chris constantinescu
f11a47ac10 RightChop instead of LeftChop
#rnx
#preflight skip

[CL 21554622 by chris constantinescu in ue5-main branch]
2022-08-24 18:35:23 -04:00
chris constantinescu
b604e668e2 Test runner:
- sleep on init to allow for sync w/ Gauntlet for Xbox platforms
- remove unused --base-global-module #fyi Stephen.Ma
Gauntlet:
- support low level tests build for different configurations (debug etc)
- readable command line arguments (-r to --reporter)
- allow argument names containing "-" e.g. --filenames-as-tags
#preflight 630536b1c00af5e2943540a1
#rb Jerome.Delattre

[CL 21550045 by chris constantinescu in ue5-main branch]
2022-08-24 15:53:16 -04:00
Devin Doucette
aea7f2af32 LowLevelTests: Allow command line arguments to the engine following "--"
The --base-global-module argument is now expected to be in the format --base-global-module=ModuleName.

RunTests has been refactored into the new private FTestRunner type.

#preflight 6303991333729407e88f8069
#preflight 6303a65a516bef57ff7420b4
#rb Chris.Constantinescu

[CL 21485276 by Devin Doucette in ue5-main branch]
2022-08-22 12:16:02 -04:00
Devin Doucette
bd378f5441 LowLevelTests: Use Unreal logging functionality instead of LocalPrint
#preflight 62ffef1cd642db47d8d52c90
#preflight 62feb1fb3c4417dcec1aaabb
#rb Chris.Constantinescu

[CL 21464557 by Devin Doucette in ue5-main branch]
2022-08-19 16:28:57 -04:00
Devin Doucette
1225bdd006 CoreUObject: Disable the FObjectHandle::Resolve test until its warnings and errors are fixed
#preflight 62ffed75904ba3bd264adf8d
#rb Zousar.Shaker
#rnx
#fyi Chris.Constantinescu

[CL 21464067 by Devin Doucette in ue5-main branch]
2022-08-19 16:12:42 -04:00
chris constantinescu
0d0ad8e4c8 Remove faulty global GBlockLocalPrint, cleanup unused VS config
All platforms preflight 62f67851b66d5d9313f240b6
#rb Devin.Doucette
#preflight 62f67866bd746abb99c0be54

[CL 21362093 by chris constantinescu in ue5-main branch]
2022-08-12 15:45:37 -04:00
chris constantinescu
1bfe2da3f2 Use platform-dependent LocalPrint and make sure output is redirected to stdout when running LowLevelTests so that local and build machine test runs print debug and fault information directly to stdout.
AssertionMacrosTest is printing out failed asserts using LocalPrint which is undesirable - implemented GBlockLocalOutput as a way to supress LocalPrint output.
Other fixes included:
- ability to select a low level test build based on any configuration, not just Development: useful for Debug builds
- New "printoutput" LLT Gauntlet parameter used for certain platforms to control output redirect to stdout
#rb David.Harvey
#preflight 62f405a05a6f2fec99b64f21

[CL 21334255 by chris constantinescu in ue5-main branch]
2022-08-11 07:31:45 -04:00
joe pribele
fc71c8cb35 [LowLevelTestsRunner] added --break to Catch command line args to break into debugger when attached by default.
#rb devin.doucette
#preflight 62f3fd00e60c9215b991bd07

[CL 21323419 by joe pribele in ue5-main branch]
2022-08-10 14:53:21 -04:00
Chris Varnsverry
0fcb315349 - Fix ConsoleListener attempting to send TCHAR* to cout, printing pointer values instead of strings.
#jira OI-4874
#review-21257422 @Michael.Kirzinger @Stephen.Ma @Alex.Kasianchuk @Sam.Zamani
#preflight 62f28751d76ea4b5030ad047

[CL 21290568 by Chris Varnsverry in ue5-main branch]
2022-08-09 12:19:20 -04:00
chris constantinescu
28702d29a5 Boilerplate
#rnx
#preflight skip

[CL 21116150 by chris constantinescu in ue5-main branch]
2022-07-15 15:53:31 -04:00
chris constantinescu
d17f975071 LLT Catch2-style macros: REQUIRE_(NO)ENSURE and CHECK_(NO)ENSURE
Based on existing Catch2 macros and reusing FDebug number of ensures counter.
#rb Scott.Lindeneau
#preflight skip

[CL 21115661 by chris constantinescu in ue5-main branch]
2022-07-15 15:23:10 -04:00
chris constantinescu
e90997b34e Catch2 - mini module cleanup
#preflight 62c793827e606620fd304713
#rb Jerome.Delattre

[CL 21004901 by chris constantinescu in ue5-main branch]
2022-07-08 06:55:54 -04:00
chris constantinescu
bf1a489842 Catch2 3.0.1
#preflight 62ba61e2dc1cd7fc47d9c708
#rnx

[CL 20857962 by chris constantinescu in ue5-main branch]
2022-06-28 10:36:46 -04:00
chris constantinescu
2300a4ad0f LLTs: Maximum log verbosity with --debug, allow logging from fault context
#rb Devin.Doucette
#preflight skip

[CL 20464451 by chris constantinescu in ue5-main branch]
2022-06-01 22:45:35 -04:00
chris constantinescu
b258362e47 Catch2 3.0 preview source code integration with on-demand static library build for all platforms.
Other preflights: Catch2 Build Library 6283d1c14316db80d194f6c8, Nightly Low Level Tests 6283d1c9114e08fa2f237e6b
#preflight 6283d1b5114e08fa2f237db3

[CL 20248814 by chris constantinescu in ue5-main branch]
2022-05-17 15:06:28 -04:00
chris constantinescu
1278a26f76 Low level tests further refactoring and test resource deployment
#preflight 626955ed0d4ac78d716eacf0
#rnx

[CL 19941830 by chris constantinescu in ue5-main branch]
2022-04-27 12:36:35 -04:00