kunit_tool's --alltests option was changed in commit
980ac3ad05 ("kunit: tool: rename all_test_uml.config, use it for --alltests")
to use a manually curated list of architecture-indpendent Kconfig
options, rather than attempting to use make allyesconfig on UML, which
was broken.
Update the kunit_tool documentation to reflect the new behaviour of
--alltests.
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The numbered list contains full path to every files that need to be
modified or created in order to implement misc-example kunit test.
Except for .kunitconfig. Which might make a newcommer confused about
where the file exists. Since there are multiple .kunitconfig files.
Fix this by using the full path to .kunitconfig.
Signed-off-by: Khalid Masum <khalid.masum.92@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Replace out-of-date external links with references to the kernel
documentation, replacing TAP webpage for the more appropriate KTAP
documentation and the UML webpage by its documentation.
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Describe the objective of the Getting Started page, which should be a
brief and beginner-friendly walkthrough for running and writing tests,
showing the reader where to find detailed instructions in other pages.
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: Maíra Canal <mairacanal@riseup.net>
Reviewed-by:Sadiya Kazi<Sadiaykazi@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The "Getting Started" guide should be beginner-friendly, therefore
add a note about the requirement of a clean source tree when running
kunit_tool for the first time, and its related error.
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Delete "kunit-tool.rst" to remove repeated info from KUnit docs.
"What is kunit_tool?" was integrated into index.rst, the remaining
sections were moved into run_wrapper.rst and renamed as follows:
"What is a .kunitconfig?" -> "Creating a ``.kunitconfig`` file"
"Getting Started with kunit_tool" -> "Running tests with kunit_tool"
"Configuring, Building, and Running Tests" ->
"Configuring, building, and running tests"
"Running Tests on QEMU" -> "Running tests on QEMU"
"Parsing Test Results" -> "Parsing test results"
"Filtering Tests" -> "Filtering tests"
"Other Useful Options" -> "Running command-line arguments"
Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: Sadiya Kazi <sadiyakazi@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
The Parameterized Testing example contains a compilation error, as the
signature for the description helper function is void(*)(const struct
sha1_test_case *, char *), and the struct is non-const. This is
warned by Clang:
error: initialization of ‘void (*)(struct sha1_test_case *, char *)’
from incompatible pointer type ‘void (*)(const struct sha1_test_case *,
char *)’ [-Werror=incompatible-pointer-types]
33 | KUNIT_ARRAY_PARAM(sha1, cases, case_to_desc);
| ^~~~~~~~~~~~
../include/kunit/test.h:1339:70: note: in definition of macro
‘KUNIT_ARRAY_PARAM’
1339 | void
(*__get_desc)(typeof(__next), char *) = get_desc; \
Signed-off-by: Maíra Canal <mairacanal@riseup.net>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Without the quoting, the example will mess up invocations like
$ run_kunit "Something with spaces"
Note: this example isn't valid, but if ever a usecase arises where a
flag argument might have spaces in it, it'll break.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Now that kunit.py's --kunitconfig is repeatable, let's create a file to
hold the various options needed to enable coverage under UML.
This can be used like so:
$ ./tools/testing/kunit/kunit.py run \
--kunitconfig=tools/testing/kunit/configs/all_tests_uml.config \
--kunitconfig=tools/testing/kunit/configs/coverage_uml.config \
--make_options=CC=/usr/bin/gcc-6
which on my system is enough to get coverage working [1].
This is still a clunky command, but far better than before.
[1] at the time of this commit, I get:
Overall coverage rate:
lines......: 11.6% (34112 of 295033 lines)
functions..: 15.3% (3721 of 24368 functions)
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Pull KUnit updates from Shuah Khan:
"Several fixes, cleanups, and enhancements to tests and framework:
- introduce _NULL and _NOT_NULL macros to pointer error checks
- rework kunit_resource allocation policy to fix memory leaks when
caller doesn't specify free() function to be used when allocating
memory using kunit_add_resource() and kunit_alloc_resource() funcs.
- add ability to specify suite-level init and exit functions"
* tag 'linux-kselftest-kunit-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (41 commits)
kunit: tool: Use qemu-system-i386 for i386 runs
kunit: fix executor OOM error handling logic on non-UML
kunit: tool: update riscv QEMU config with new serial dependency
kcsan: test: use new suite_{init,exit} support
kunit: tool: Add list of all valid test configs on UML
kunit: take `kunit_assert` as `const`
kunit: tool: misc cleanups
kunit: tool: minor cosmetic cleanups in kunit_parser.py
kunit: tool: make parser stop overwriting status of suites w/ no_tests
kunit: tool: remove dead parse_crash_in_log() logic
kunit: tool: print clearer error message when there's no TAP output
kunit: tool: stop using a shell to run kernel under QEMU
kunit: tool: update test counts summary line format
kunit: bail out of test filtering logic quicker if OOM
lib/Kconfig.debug: change KUnit tests to default to KUNIT_ALL_TESTS
kunit: Rework kunit_resource allocation policy
kunit: fix debugfs code to use enum kunit_status, not bool
kfence: test: use new suite_{init/exit} support, add .kunitconfig
kunit: add ability to specify suite-level init and exit functions
kunit: rename print_subtest_{start,end} for clarity (s/subtest/suite)
...
Pull documentation updates from Jonathan Corbet:
"It was a moderately busy cycle for documentation; highlights include:
- After a long period of inactivity, the Japanese translations are
seeing some much-needed maintenance and updating.
- Reworked IOMMU documentation
- Some new documentation for static-analysis tools
- A new overall structure for the memory-management documentation.
This is an LSFMM outcome that, it is hoped, will help encourage
developers to fill in the many gaps. Optimism is eternal...but
hopefully it will work.
- More Chinese translations.
Plus the usual typo fixes, updates, etc"
* tag 'docs-5.19' of git://git.lwn.net/linux: (70 commits)
docs: pdfdocs: Add space for chapter counts >= 100 in TOC
docs/zh_CN: Add dev-tools/gdb-kernel-debugging.rst Chinese translation
input: Docs: correct ntrig.rst typo
input: Docs: correct atarikbd.rst typos
MAINTAINERS: Become the docs/zh_CN maintainer
docs/zh_CN: fix devicetree usage-model translation
mm,doc: Add new documentation structure
Documentation: drop more IDE boot options and ide-cd.rst
Documentation/process: use scripts/get_maintainer.pl on patches
MAINTAINERS: Add entry for DOCUMENTATION/JAPANESE
docs/trans/ja_JP/howto: Don't mention specific kernel versions
docs/ja_JP/SubmittingPatches: Request summaries for commit references
docs/ja_JP/SubmittingPatches: Add Suggested-by as a standard signature
docs/ja_JP/SubmittingPatches: Randy has moved
docs/ja_JP/SubmittingPatches: Suggest the use of scripts/get_maintainer.pl
docs/ja_JP/SubmittingPatches: Update GregKH links
Documentation/sysctl: document max_rcu_stall_to_panic
Documentation: add missing angle bracket in cgroup-v2 doc
Documentation: dev-tools: use literal block instead of code-block
docs/zh_CN: add vm numa translation
...
KUnit has support for setup/cleanup logic for each test case in a suite.
But it lacks the ability to specify setup/cleanup for the entire suite
itself.
This can be used to do setup that is too expensive or cumbersome to do
for each test.
Or it can be used to do simpler things like log debug information after
the suite completes.
It's a fairly common feature, so the lack of it is noticeable.
Some examples in other frameworks and languages:
* https://docs.python.org/3/library/unittest.html#setupclass-and-teardownclass
* https://google.github.io/googletest/reference/testing.html#Test::SetUpTestSuite
Meta:
This is very similar to this patch here: https://lore.kernel.org/linux-kselftest/20210805043503.20252-3-bvanassche@acm.org/
The changes from that patch:
* pass in `struct kunit *` so users can do stuff like
`kunit_info(suite, "debug message")`
* makes sure the init failure is bubbled up as a failure
* updates kunit-example-test.c to use a suite init
* Updates kunit/usage.rst to mention the new support
* some minor cosmetic things
* use `suite_{init,exit}` instead of `{init/exit}_suite`
* make suite init error message more consistent w/ test init
* etc.
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Recent changes have made it so the current set is not sufficient.
Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.
Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's
hopefully less fragile (esp. given we're tied to GCC 6 and lower).
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Tested-by: Maxime Ripard <maxime@cerno.tech>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Background:
Currently, a reader looking at kunit/test.h will find the file is quite
long, and the first meaty comment is a doc comment about struct
kunit_resource.
Most users will not ever use the KUnit resource API directly.
They'll use kunit_kmalloc() and friends, or decide it's simpler to do
cleanups via labels (it often can be) instead of figuring out how to use
the API.
It's also logically separate from everything else in test.h.
Removing it from the file doesn't cause any compilation errors (since
struct kunit has `struct list_head resources` to store them).
This commit:
Let's move it into a kunit/resource.h file and give it a separate page
in the docs, kunit/api/resource.rst.
We include resource.h at the bottom of test.h since
* don't want to force existing users to add a new include if they use the API
* it accesses `lock` inside `struct kunit` in a inline func
* so we can't just forward declare, and the alternatives require
uninlining the func, adding hepers to lock/unlock, or other more
invasive changes.
Now the first big comment in test.h is about kunit_case, which is a lot
more relevant to what a new user wants to know.
A side effect of this is git blame won't properly track history by
default, users need to run
$ git blame -L ,1 -C17 include/kunit/resource.h
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Commit ddbd60c779 ("kunit: use --build_dir=.kunit as default") changed
the default --build_dir, which had the side effect of making
`.kunitconfig` move to `.kunit/.kunitconfig`.
However, the first few lines of kunit/start.rst never got updated, oops.
Fix this by telling people to run kunit.py first, which will
automatically generate the .kunit directory and .kunitconfig file, and
then edit the file manually as desired.
Reported-by: Yifan Yuan <alpc_metic@live.com>
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Pull more documentation updates from Jonathan Corbet:
"Some late-arriving documentation improvements.
This is mostly build-system fixes from Mauro and Akira; I also took
the liberty of dropping in my 'messy diffstat' document"
* tag 'docs-5.18-2' of git://git.lwn.net/linux:
docs: Add a document on how to fix a messy diffstat
docs: sphinx/requirements: Limit jinja2<3.1
Documentation: kunit: Fix cross-referencing warnings
scripts/kernel-doc: change the line number meta info
scripts/get_abi: change the file/line number meta info
docs: kernel_include.py: add sphinx build dependencies
docs: kernel_abi.py: add sphinx build dependencies
docs: kernel_feat.py: add build dependencies
scripts/get_feat.pl: allow output the parsed file names
docs: kfigure.py: Don't warn of missing PDF converter in 'make htmldocs'
Documentation: Fix duplicate statement about raw_spinlock_t type
