The error path of copying the old config used the wrong variable in the
error message:
$ mkdir /tmp/build
$ ./tools/testing/ktest/config-bisect.pl -b /tmp/build config-good /tmp/config-bad
$ chmod 0 /tmp/build
$ ./tools/testing/ktest/config-bisect.pl -b /tmp/build config-good /tmp/config-bad good
cp /tmp/build//.config config-good.tmp ... [0 seconds] FAILED!
Use of uninitialized value $config in concatenation (.) or string at ./tools/testing/ktest/config-bisect.pl line 744.
failed to copy to config-good.tmp
When it should have shown:
failed to copy /tmp/build//.config to config-good.tmp
Cc: stable@vger.kernel.org
Cc: John 'Warthog9' Hawley <warthog9@kernel.org>
Fixes: 0f0db06599 ("ktest: Add standalone config-bisect.pl program")
Link: https://patch.msgid.link/20251203180924.6862bd26@gandalf.local.home
Reported-by: "John W. Krahn" <jwkrahn@shaw.ca>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When testing a series of commits that also includes changes to the Linux
tools directory, it is useless to test the changes in tools as they may
not affect the kernel itself. Doing tests on the kernel for changes that
do not affect the kernel is a waste of time.
Add a PATCHCHECK_SKIP that takes a series of shas that will be skipped
while doing the individual commit tests.
For example, the runtime verification may have a series of commits like:
$ git log --abbrev-commit --pretty=oneline fac5493251a6~1..HEAD
3d3800b4f7 rv: Remove rv_reactor's reference counter
3d3c376118 rv: Merge struct rv_reactor_def into struct rv_reactor
24cbfe18d5 rv: Merge struct rv_monitor_def into struct rv_monitor
b0c08dd534 rv: Remove unused field in struct rv_monitor_def
58d5f0d437 (debiantesting-x86-64/trace/rv/core) rv: Return init error when registering monitors
560473f2e2 verification/rvgen: Organise Kconfig entries for nested monitors
9efcf59082 tools/dot2c: Fix generated files going over 100 column limit
1160ccaf77 tools/rv: Stop gracefully also on SIGTERM
f60227f344 tools/rv: Do not skip idle in trace
f3735df628 verification/rvgen: Do not generate unused variables
6fb37c2a27 verification/rvgen: Generate each variable definition only once
8cfcf9b0e9 verification/rvgen: Support the 'next' operator
fac5493251 rv: Allow to configure the number of per-task monitor
Where the first commit touches the kernel followed by a series of commits
that do not, and ends with commits that do. Instead of having to add
multiple patchcheck tests to handle the gaps, just include the commits
that should not be tested:
$ git log --abbrev-commit --pretty=oneline fac5493251a6~1..HEAD |
grep -e verification -e tools/ | cut -d' ' -f1 |
while read a ; do echo -n "$a "; done
560473f2e29efcf590821160ccaf77f60227f344f3735df6286fb37c2a278cfcf9b0e9
Then set PATCHCHECK_SKIP to that, and those commits will be skipped.
PATCHCHECK_SKIP = 560473f2e29efcf590821160ccaf77f60227f344f3735df6286fb37c2a278cfcf9b0e9
Cc: John 'Warthog9' Hawley <warthog9@kernel.org>
Cc: Dhaval Giani <dhaval.giani@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/20250725112153.1dd06b84@gandalf.local.home
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
As ktest.pl can run in various different directories, to make sure the
test is running in the proper directory with the proper source and proper
destination directory, display the content of BUILD_DIR and OUTPUT_DIR at
the start of every test.
This can be helpful for the test runner to stop the test if a test is
running in the wrong location instead of finding out after the test has
completed.
Cc: "John Warthog9 Hawley" <warthog9@kernel.org>
Cc: Dhaval Giani <dhaval.giani@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/20250718202053.898022631@kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If a default variable contains itself, do not recurse on it.
For example:
ADD_CONFIG := ${CONFIG_DIR}/temp_config
DEFAULTS
ADD_CONFIG = ${CONFIG_DIR}/default_config ${ADD_CONFIG}
The above works because the temp variable ADD_CONFIG (is a temp because it
is created with ":=") is already defined, it will be substituted in the
variable option. But if it gets commented out:
# ADD_CONFIG := ${CONFIG_DIR}/temp_config
DEFAULTS
ADD_CONFIG = ${CONFIG_DIR}/default_config ${ADD_CONFIG}
Then the above will go into a recursive loop where ${ADD_CONFIG} will
get replaced with the current definition of ADD_CONFIG which contains the
${ADD_CONFIG} and that will also try to get converted. ktest.pl will error
after 100 attempts of recursion and fail.
When replacing a variable with the default variable, if the default
variable contains itself, do not replace it.
Cc: "John Warthog9 Hawley" <warthog9@kernel.org>
Cc: Dhaval Giani <dhaval.giani@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/20250718202053.732189428@kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add -D option that lets the user override options in the config.
For instance, if the config has: BUILD_NOCLEAN=1 which prevents mrproper
from being called before builds, and the user wants to call it once. The
user can run:
ktest -D BUILD_NOCLEAN=0 config
And the default "BUILD_NOCLEAN" options will be disabled.
If the user wants to change the second test to do a build and not boot,
the user can run:
ktest -D 'TEST_TYPE[2]=build' config
Where the '[#]' is for the test to assign the variable for. In the above
example, it will happen on test 2.
Cc: "John Warthog9 Hawley" <warthog9@kernel.org>
Cc: Dhaval Giani <dhaval.giani@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/20250718202053.231478909@kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
In the case of a test that uses the special option ${KERNEL_VERSION} in one
of its settings but has no configuration available in ${OUTPUT_DIR}, for
example if it's a new empty directory, then the `make kernelrelease` call
will fail and the subroutine will chomp an empty string, silently. Fix that
by adding an empty configuration and retrying.
Cc: stable@vger.kernel.org
Cc: John Hawley <warthog9@eaglescrag.net>
Fixes: 5f9b6ced04 ("ktest: Bisecting, install modules, add logging")
Link: https://lore.kernel.org/20241205-ktest_kver_fallback-v2-1-869dae4c7777@suse.com
Signed-off-by: Ricardo B. Marliere <rbm@suse.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Some distros have grub2 config files with the lines
if [ x"${feature_menuentry_id}" = xy ]; then
menuentry_id_option="--id"
else
menuentry_id_option=""
fi
which match the skip regex defined for grub2 in get_grub_index():
$skip = '^\s*menuentry';
These false positives cause the grub number to be higher than it
should be, and the wrong kernel can end up booting.
Grub documents the menuentry command with whitespace between it and the
title, so make the skip regex reflect this.
Link: https://lore.kernel.org/20240904175530.84175-1-daniel.m.jordan@oracle.com
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Acked-by: John 'Warthog9' Hawley (Tenstorrent) <warthog9@eaglescrag.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If a warning happens at build, give a warning at the end:
Build time: 1 minute 40 seconds
Install time: 17 seconds
Reboot time: 25 seconds
*** WARNING found in build: 1 ***
*******************************************
*******************************************
KTEST RESULT: TEST 1 SUCCESS!!!! **
*******************************************
*******************************************
This way, even if the test isn't made to fail on warnings during the
build, a message is still displayed that warnings were found.
Link: https://lore.kernel.org/<20240819172028.3a7fae09@gandalf.local.home>
Acked-by: John 'Warthog9' Hawley (Tenstorrent) <warthog9@eaglescrag.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The test type "make_warnings_file" should have no mandatory configuration
parameters other than the ones required by the "build" test type, because
its purpose is to create a file with build warnings that may or may not be
used by other subsequent tests. Currently, the only way to use it as a
stand-alone test is by setting POWER_CYCLE, CONSOLE, SSH_USER,
BUILD_TARGET, TARGET_IMAGE, REBOOT_TYPE and GRUB_MENU.
Link: https://lkml.kernel.org/r/20240315-ktest-v2-1-c5c20a75f6a3@marliere.net
Cc: John Hawley <warthog9@eaglescrag.net>
Signed-off-by: Ricardo B. Marliere <ricardo@marliere.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Allow a variable to contain another variable. This will allow the
${shell <command>} to have its command include variables.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
There is a disconnect between the run_command function and the
wait_for_input. The wait_for_input has a default timeout of 2 minutes. But
if that happens, the run_command loop will exit out to the waitpid() of
the executing command. This fails in that it no longer monitors the
command, and also, the ssh to the test box can hang when its finished, as
it's waiting for the pipe it's writing to to flush, but the loop that
reads that pipe has already exited, leaving the command stuck, and the
test hangs.
Instead, make the default "wait_for_input" of the run_command infinite,
and allow the user to override it if they want with a default timeout
option "RUN_TIMEOUT".
But this fixes the hang that happens when the pipe is full and the ssh
session never exits.
Cc: stable@vger.kernel.org
Fixes: 6e98d1b441 ("ktest: Add timeout to ssh command")
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When monitoring the console output, the stdout is being redirected to do
so. If Ctrl^C is hit during this mode, the stdout is not back to the
console, the user does not see anything they type (no echo).
Add "end_monitor" to the SIGINT interrupt handler to give back the console
on Ctrl^C.
Cc: stable@vger.kernel.org
Fixes: 9f2cdcbbb9 ("ktest: Give console process a dedicated tty")
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
In the "reboot" command, it does a check of the machine to see if it is
still alive with a simple "ssh echo" command. If it fails, it will assume
that a normal "ssh reboot" is not possible and force a power cycle.
In this case, the "start_monitor" is executed, but the "end_monitor" is
not, and this causes the screen will not be given back to the console. That
is, after the test, a "reset" command needs to be performed, as "echo" is
turned off.
Cc: stable@vger.kernel.org
Fixes: 6474ace999 ("ktest.pl: Powercycle the box on reboot if no connection can be made")
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Allow variables to execute shell commands. Note, these are processed when
they are first seen while parsing the config file. This is useful if you
have the same config file used for multiple hosts (as they may be in a git
repository).
HOSTNAME := ${shell hostname}
DEFAULTS IF "${HOSTNAME}" == "frodo"
Link: https://lkml.kernel.org/r/20221207212944.277ee850@gandalf.local.home
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
grub2 has submenus where to use grub-reboot, it requires:
grub-reboot X>Y
where X is the main index and Y is the submenu. Thus if you have:
menuentry 'Debian GNU/Linux' --class debian --class gnu-linux ...
[...]
}
submenu 'Advanced options for Debian GNU/Linux' $menuentry_id_option ...
menuentry 'Debian GNU/Linux, with Linux 6.0.0-4-amd64' --class debian --class gnu-linux ...
[...]
}
menuentry 'Debian GNU/Linux, with Linux 6.0.0-4-amd64 (recovery mode)' --class debian --class gnu-linux ...
[...]
}
menuentry 'Debian GNU/Linux, with Linux test' --class debian --class gnu-linux ...
[...]
}
And wanted to boot to the "Linux test" kernel, you need to run:
# grub-reboot 1>2
As 1 is the second top menu (the submenu) and 2 is the third of the sub
menu entries.
Have the grub.cfg parsing for grub2 handle such cases.
Cc: stable@vger.kernel.org
Fixes: a15ba91361 ("ktest: Add support for grub2")
Reviewed-by: John 'Warthog9' Hawley (VMware) <warthog9@eaglescrag.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
After a full run of a make_min_config test, I noticed there were a lot of
CONFIGs still enabled that really should not be. Looking at them, I
noticed they were all defined as "default y". The issue is that the test
simple removes the config and re-runs make oldconfig, which enables it
again because it is set to default 'y'. Instead, explicitly disable the
config with writing "# CONFIG_FOO is not set" to the file to keep it from
being set again.
With this change, one of my box's minconfigs went from 768 configs set,
down to 521 configs set.
Link: https://lkml.kernel.org/r/20221202115936.016fce23@gandalf.local.home
Cc: stable@vger.kernel.org
Fixes: 0a05c769a9 ("ktest: Added config_bisect test type")
Reviewed-by: John 'Warthog9' Hawley (VMware) <warthog9@eaglescrag.net>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
bootconfig is a new feature that appends scripts onto the initrd, and the
kernel executes the scripts as an extended kernel command line.
Need to add tests to test that the happened. To test the bootconfig
properly, the initrd needs to be updated and the kernel rebooted. ktest is
the perfect solution to perform these tests.
Add a example bootconfig.conf in the tools/testing/ktest/examples/include
and example bootconfig scripts in tools/testing/ktest/examples/bootconfig
and also include verifier scripts that ktest will install on the target
and run to make sure that the bootconfig options in the scripts took place
after the target rebooted with the new initrd update.
Link: https://lkml.kernel.org/r/20210618112647.6a81dec5@oasis.local.home
Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Steven Rostedt
Ayush Jain
WangYuli
Ba Jing
Ricardo B. Marliere
Daniel Jordan
Ricardo B. Marliere
Masami Hiramatsu (Google)