The Perl localtime() function returns the month starting at 0 not 1. This
caused the date produced to create the directory for saving files of a
failed run to have the month off by one.
machine-test-useconfig-fail-20260314073628
The above happened in April, not March. The correct name should have been:
machine-test-useconfig-fail-20260414073628
This was somewhat confusing.
Cc: stable@vger.kernel.org
Cc: John 'Warthog9' Hawley <warthog9@kernel.org>
Link: https://patch.msgid.link/20260420142426.33ad0293@fedora
Fixes: 7faafbd696 ("ktest: Add open and close console and start stop monitor")
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
STORE_FAILURES was only saved from fail(), so paths that reached dodie()
could exit without preserving failure logs.
That includes fatal hook paths such as:
POST_BUILD_DIE = 1
and ordinary failures when:
DIE_ON_FAILURE = 1
Call save_logs("fail", ...) from dodie() too so fatal failures keep the
same STORE_FAILURES artifacts as non-fatal fail() paths.
Cc: John Hawley <warthog9@eaglescrag.net>
Link: https://patch.msgid.link/20260318-ktest-fixes-v1-1-9dd94d46d84c@suse.com
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When working on a ktest configuration, it is often useful to inspect the
final option values after includes, defaults, per-test overrides, and
variable expansion have been applied, without actually starting a test run.
Add a --dry-run option that reads the configuration, prints the test
preamble using resolved option values, and exits before opening LOG_FILE or
executing any test logic.
This is useful for debugging ktest configurations and for scripts that need
to validate the final resolved settings without triggering side effects.
Cc: John Hawley <warthog9@eaglescrag.net>
Cc: Andrea Righi <arighi@nvidia.com>
Cc: Marcos Paulo de Souza <mpdesouza@suse.com>
Cc: Matthieu Baerts <matttbe@kernel.org>
Cc: Fernando Fernandez Mancera <fmancera@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Link: https://patch.msgid.link/20260307-ktest-fixes-v1-9-565d412f4925@suse.com
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
PRE_KTEST can be useful for setting up the environment and POST_KTEST to
tear it down, however POST_KTEST only runs on the normal end-of-run path.
It is skipped when ktest exits through dodie() or cancel_test(). Final
cleanup hooks are skipped.
Factor the final hook execution into run_post_ktest(), call it from the
normal exit path and from the early exit paths, and guard it so the hook
runs at most once.
Cc: John Hawley <warthog9@eaglescrag.net>
Cc: Andrea Righi <arighi@nvidia.com>
Cc: Marcos Paulo de Souza <mpdesouza@suse.com>
Cc: Matthieu Baerts <matttbe@kernel.org>
Cc: Fernando Fernandez Mancera <fmancera@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Link: https://patch.msgid.link/20260307-ktest-fixes-v1-8-565d412f4925@suse.com
Fixes: 921ed4c720 ("ktest: Add PRE/POST_KTEST and TEST options")
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The POWER_CYCLE fallback added to reboot() flushes monitor output at the
wrong time. In the untimed reboot path, flushing immediately after
start_monitor() can consume the first output from the new boot before
monitor() begins reading it. In the timed path, flushing after POWER_CYCLE
can eat the "Linux version" banner or REBOOT_SUCCESS_LINE from the new
kernel.
That makes ktest miss the boot it is waiting for and can trigger an
unnecessary second power cycle.
Start the monitor before POWER_CYCLE so the reference counting stays
balanced, but only flush when reboot() was asked to wait for a timed
reboot. Perform that flush before issuing POWER_CYCLE so it drains stale
output from the old kernel instead of consuming the next boot.
Cc: John Hawley <warthog9@eaglescrag.net>
Cc: Andrea Righi <arighi@nvidia.com>
Cc: Marcos Paulo de Souza <mpdesouza@suse.com>
Cc: Matthieu Baerts <matttbe@kernel.org>
Cc: Fernando Fernandez Mancera <fmancera@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Link: https://patch.msgid.link/20260307-ktest-fixes-v1-6-565d412f4925@suse.com
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
A per-test override can clear an inherited default option by assigning an
empty value, but __set_test_option() still used option_defined() to decide
whether a per-test key existed. That turned an empty per-test assignment
back into "fall back to the default", so tests still could not clear
inherited settings.
For example:
DEFAULTS
(...)
LOG_FILE = /tmp/ktest-empty-override.log
CLEAR_LOG = 1
ADD_CONFIG = /tmp/.config
TEST_START
TEST_TYPE = build
BUILD_TYPE = nobuild
ADD_CONFIG =
This would run the test with ADD_CONFIG[1] = /tmp/.config
Fix by checking whether the per-test key exists before falling back. If it
does exist but is empty, treat it as unset for that test and stop the
fallback chain there.
Cc: John Hawley <warthog9@eaglescrag.net>
Cc: Andrea Righi <arighi@nvidia.com>
Cc: Marcos Paulo de Souza <mpdesouza@suse.com>
Cc: Matthieu Baerts <matttbe@kernel.org>
Cc: Fernando Fernandez Mancera <fmancera@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Link: https://patch.msgid.link/20260307-ktest-fixes-v1-4-565d412f4925@suse.com
Fixes: 22c37a9ac4 ("ktest: Allow tests to undefine default options")
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Config variables are expanded when they are assigned. A first-time append
such as:
VAR := ${VAR} foo
leaves the literal ${VAR} in the stored value because VAR has not been
defined yet. Later expansions then carry the self-reference forward instead
of behaving like an empty prefix.
Drop an unescaped self-reference when the variable has no current value,
and trim the outer whitespace left behind. Keep escaped \${VAR} references
unchanged so literal text still works.
Cc: John Hawley <warthog9@eaglescrag.net>
Cc: Andrea Righi <arighi@nvidia.com>
Cc: Marcos Paulo de Souza <mpdesouza@suse.com>
Cc: Matthieu Baerts <matttbe@kernel.org>
Cc: Fernando Fernandez Mancera <fmancera@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Link: https://patch.msgid.link/20260307-ktest-fixes-v1-3-565d412f4925@suse.com
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
LOG_FILE is expanded immediately after the config file is parsed with
eval_option(..., -1). That uses the default context, not the same option
resolution path used for tests. If LOG_FILE depends on options that are
finalized per test, it can be resolved from stale values before the first
test starts.
Resolve LOG_FILE through set_test_option("LOG_FILE", 1) instead so it uses
the same expansion rules as the rest of the test options.
Cc: John Hawley <warthog9@eaglescrag.net>
Cc: Andrea Righi <arighi@nvidia.com>
Cc: Marcos Paulo de Souza <mpdesouza@suse.com>
Cc: Matthieu Baerts <matttbe@kernel.org>
Cc: Fernando Fernandez Mancera <fmancera@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Link: https://patch.msgid.link/20260307-ktest-fixes-v1-2-565d412f4925@suse.com
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When testing a series of commits that also includes changes to the Linux
tools directory, it is useless to test the changes in tools as they may
not affect the kernel itself. Doing tests on the kernel for changes that
do not affect the kernel is a waste of time.
Add a PATCHCHECK_SKIP that takes a series of shas that will be skipped
while doing the individual commit tests.
For example, the runtime verification may have a series of commits like:
$ git log --abbrev-commit --pretty=oneline fac5493251a6~1..HEAD
3d3800b4f7 rv: Remove rv_reactor's reference counter
3d3c376118 rv: Merge struct rv_reactor_def into struct rv_reactor
24cbfe18d5 rv: Merge struct rv_monitor_def into struct rv_monitor
b0c08dd534 rv: Remove unused field in struct rv_monitor_def
58d5f0d437 (debiantesting-x86-64/trace/rv/core) rv: Return init error when registering monitors
560473f2e2 verification/rvgen: Organise Kconfig entries for nested monitors
9efcf59082 tools/dot2c: Fix generated files going over 100 column limit
1160ccaf77 tools/rv: Stop gracefully also on SIGTERM
f60227f344 tools/rv: Do not skip idle in trace
f3735df628 verification/rvgen: Do not generate unused variables
6fb37c2a27 verification/rvgen: Generate each variable definition only once
8cfcf9b0e9 verification/rvgen: Support the 'next' operator
fac5493251 rv: Allow to configure the number of per-task monitor
Where the first commit touches the kernel followed by a series of commits
that do not, and ends with commits that do. Instead of having to add
multiple patchcheck tests to handle the gaps, just include the commits
that should not be tested:
$ git log --abbrev-commit --pretty=oneline fac5493251a6~1..HEAD |
grep -e verification -e tools/ | cut -d' ' -f1 |
while read a ; do echo -n "$a "; done
560473f2e29efcf590821160ccaf77f60227f344f3735df6286fb37c2a278cfcf9b0e9
Then set PATCHCHECK_SKIP to that, and those commits will be skipped.
PATCHCHECK_SKIP = 560473f2e29efcf590821160ccaf77f60227f344f3735df6286fb37c2a278cfcf9b0e9
Cc: John 'Warthog9' Hawley <warthog9@kernel.org>
Cc: Dhaval Giani <dhaval.giani@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/20250725112153.1dd06b84@gandalf.local.home
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
As ktest.pl can run in various different directories, to make sure the
test is running in the proper directory with the proper source and proper
destination directory, display the content of BUILD_DIR and OUTPUT_DIR at
the start of every test.
This can be helpful for the test runner to stop the test if a test is
running in the wrong location instead of finding out after the test has
completed.
Cc: "John Warthog9 Hawley" <warthog9@kernel.org>
Cc: Dhaval Giani <dhaval.giani@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/20250718202053.898022631@kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If a default variable contains itself, do not recurse on it.
For example:
ADD_CONFIG := ${CONFIG_DIR}/temp_config
DEFAULTS
ADD_CONFIG = ${CONFIG_DIR}/default_config ${ADD_CONFIG}
The above works because the temp variable ADD_CONFIG (is a temp because it
is created with ":=") is already defined, it will be substituted in the
variable option. But if it gets commented out:
# ADD_CONFIG := ${CONFIG_DIR}/temp_config
DEFAULTS
ADD_CONFIG = ${CONFIG_DIR}/default_config ${ADD_CONFIG}
Then the above will go into a recursive loop where ${ADD_CONFIG} will
get replaced with the current definition of ADD_CONFIG which contains the
${ADD_CONFIG} and that will also try to get converted. ktest.pl will error
after 100 attempts of recursion and fail.
When replacing a variable with the default variable, if the default
variable contains itself, do not replace it.
Cc: "John Warthog9 Hawley" <warthog9@kernel.org>
Cc: Dhaval Giani <dhaval.giani@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/20250718202053.732189428@kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add -D option that lets the user override options in the config.
For instance, if the config has: BUILD_NOCLEAN=1 which prevents mrproper
from being called before builds, and the user wants to call it once. The
user can run:
ktest -D BUILD_NOCLEAN=0 config
And the default "BUILD_NOCLEAN" options will be disabled.
If the user wants to change the second test to do a build and not boot,
the user can run:
ktest -D 'TEST_TYPE[2]=build' config
Where the '[#]' is for the test to assign the variable for. In the above
example, it will happen on test 2.
Cc: "John Warthog9 Hawley" <warthog9@kernel.org>
Cc: Dhaval Giani <dhaval.giani@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/20250718202053.231478909@kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
In the case of a test that uses the special option ${KERNEL_VERSION} in one
of its settings but has no configuration available in ${OUTPUT_DIR}, for
example if it's a new empty directory, then the `make kernelrelease` call
will fail and the subroutine will chomp an empty string, silently. Fix that
by adding an empty configuration and retrying.
Cc: stable@vger.kernel.org
Cc: John Hawley <warthog9@eaglescrag.net>
Fixes: 5f9b6ced04 ("ktest: Bisecting, install modules, add logging")
Link: https://lore.kernel.org/20241205-ktest_kver_fallback-v2-1-869dae4c7777@suse.com
Signed-off-by: Ricardo B. Marliere <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Some distros have grub2 config files with the lines
if [ x"${feature_menuentry_id}" = xy ]; then
menuentry_id_option="--id"
else
menuentry_id_option=""
fi
which match the skip regex defined for grub2 in get_grub_index():
$skip = '^\s*menuentry';
These false positives cause the grub number to be higher than it
should be, and the wrong kernel can end up booting.
Grub documents the menuentry command with whitespace between it and the
title, so make the skip regex reflect this.
Link: https://lore.kernel.org/20240904175530.84175-1-daniel.m.jordan@oracle.com
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Acked-by: John 'Warthog9' Hawley (Tenstorrent) <warthog9@eaglescrag.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If a warning happens at build, give a warning at the end:
Build time: 1 minute 40 seconds
Install time: 17 seconds
Reboot time: 25 seconds
*** WARNING found in build: 1 ***
*******************************************
*******************************************
KTEST RESULT: TEST 1 SUCCESS!!!! **
*******************************************
*******************************************
This way, even if the test isn't made to fail on warnings during the
build, a message is still displayed that warnings were found.
Link: https://lore.kernel.org/<20240819172028.3a7fae09@gandalf.local.home>
Acked-by: John 'Warthog9' Hawley (Tenstorrent) <warthog9@eaglescrag.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The test type "make_warnings_file" should have no mandatory configuration
parameters other than the ones required by the "build" test type, because
its purpose is to create a file with build warnings that may or may not be
used by other subsequent tests. Currently, the only way to use it as a
stand-alone test is by setting POWER_CYCLE, CONSOLE, SSH_USER,
BUILD_TARGET, TARGET_IMAGE, REBOOT_TYPE and GRUB_MENU.
Link: https://lkml.kernel.org/r/20240315-ktest-v2-1-c5c20a75f6a3@marliere.net
Cc: John Hawley <warthog9@eaglescrag.net>
Signed-off-by: Ricardo B. Marliere <ricardo@marliere.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
