Add '=~' and '!~' to the list of allowed conditionals for DEFAULT and
TEST_START section if statements.
ie.
TEST_START IF TEST =~ .*test$
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The option IGNORE_ERRORS is used to allow a test to succeed even if a
warning appears from the kernel. Sometimes kernels will produce warnings
that are not associated with a test, and the user wants to test
something else.
The IGNORE_ERRORS works for boot up, but was not preventing test runs to
succeed if the kernel produced a warning.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The min configs are saved in a perl hash called force_configs, and this
hash is used to add configs to the .config file. But it was not being
reset between tests and a min config from a previous test would affect
the min config of the next test causing undesirable results.
Reset the force_config hash at the start of each test.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Usually the target is booted into a dependable kernel when a test
starts. The test will install the test kernel and reboot the box. But
there may be a time that the kernel is running an unreliable kernel and
the reboot may crash.
Have ktest detect crashes on a reboot and force a power-cycle instead.
This can usually happen if a test kernel was installed to run manual
tests, but the user forgot to reboot to the known good kernel.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If the console is constantly outputting content, this can cause ktest
to get stuck waiting on the monitor to settle down.
The option MAX_MONITOR_WAIT is the maximum time (in seconds) for ktest
to wait for the console to flush.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
With a name like 'oldnoconfig' one may think that the config generated
would disable all configs that were not defined (selecting "no" for all
options). But this is not the case. It selects the default. If a config
has a 'default y', then it is added if not specified.
This broke the config bisect, because options not specified by a config
will just use the default, where it expected to turn off. This caused an
option to be enabled that disabled an option that would break the build.
The end result was that we never found the bad config at the end of the
test.
Instead of using 'make oldnoconfig', ktest now builds the options it
expects enabled and disabled. When it turns off an option, it will no
longer remove it, but actually set it to:
# CONFIG_FOO is not set.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The config-bisect can take a bad config and bisect it down to find out
what config actually breaks the config. But as all tests will apply a
minconfig (defined by a user) to apply before booting, it is possible
that the minconfig could actually make the bad config work (minconfigs
can disable configs). The end result is that the config bisect test will
not find a config that breaks. This can be rather frustrating to the
user.
The CONFIG_BISECT_CHECK option, when set to 1, will make sure that the
bad config (with the minconfig applied) still fails before trying to
bisect.
And yes, I did get burned by this.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add the PRE_INSTALL option that will allow a user to specify a shell
command to be executed before the install operation executes.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
In order to let the user add commands before and after ktest runs, the
PRE_KTEST and POST_KTEST options are defined. They hold shell commands
that will execute befor ktest runs its first test, as well as when it
completed its last test.
The PRE_TEST and POST_TEST will be run befor and after (respectively)
for a given test. They can either be global (done for all tests) or
defined by a single test.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add a README that explains what the different example configs in the
ktest example directory are about.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
I used the snowball.conf in a live demo that demonstrated how to use
ktest.pl with a snowball ARM board. I've been asked to included that
config in the ktest repository.
Here it is.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add the config that I use to test several archs. I downloaded several
cross compilers from:
http://kernel.org/pub/tools/crosstool/files/bin/x86_64/
and this config is an example to crosscompile several archs to make sure
that your changes do not break archs that you are not working on.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
I've been asked several times to provide more useful example configs for
ktest.pl, as the sample.conf is too complex (because it explains all
configs). This adds configs broken up by use case, and these configs are
based on actual configs that I use on a daily basis.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If the file that OUTPUT_MIN_CONFIG exists then ktest.pl will prompt the
user and ask them if the OUTPUT_MIN_CONFIG should be used as the
starting point for make_min_config instead of MIN_CONFIG.
This is usually the case, and to allow the user to do so, which is
helpful if the user is creating different min configs based on tests,
and they know one is a superset of another test, they can set
USE_OUTPUT_MIN_CONFIG to one, which will prevent kest.pl from prompting
to use the OUTPUT_MIN_CONFIG and it will just use it.
If USE_OUTPUT_MIN_CONIFG is set to zero, then ktest.pl will continue to
use MIN_CONFIG instead.
The default is that USE_OUTPUT_MIN_CONFIG is undefined.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add a MIN_CONFIG_TYPE that can be set to 'test' or 'boot'. The default
is 'boot' which is what make_min_config has done previously: makes a
config file that is the minimum needed to boot the target.
But when MIN_CONFIG_TYPE is set to 'test', not only must the target
boot, but it must also successfully run the TEST. This allows the
creation of a config file that is the minimum to boot and also
perform ssh to the target, or anything else a developer wants.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The PRE_BUILD and POST_BUILD options of ktest are added to allow the
user to add temporary patch to the system and remove it on builds. This
is sometimes use to take a change from another git branch and add it to
a series without the fix so that this series can be tested, when an
unrelated bug exists in the series.
The problem comes when a tagged commit is being used. For example, if
v3.2 is being tested, and we add a patch to it, the kernelrelease for
that commit will be 3.2.0+, but without the patch the version will be
3.2.0. This can cause problems when the kernelrelease is determined for
creating the /lib/modules directory. The kernel booting has the '+' but
the module directory will not, and the modules will be missing for that
boot, and may not allow the kernel to succeed.
The fix is to put the creation of the kernelrelease in the POST_BUILD
logic, before it applies the POST_BUILD operation. The POST_BUILD is
where the patch may be removed, removing the '+' from the kernelrelease.
The calculation of the kernelrelease will also stay in its current
location but will be ignored if it was already calculated previously.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The change to let individual tests decide to reboot the machine on
success of the entire test also prevented errors from rebooting
when an error was detected.
The "no_reboot" variable was only cleared if the test had
reboot_on_success set. But the no_reboot variable also prevents the test
rebooting when an error was detected even when REBOOT_ON_ERROR was set.
Add a new "reboot_success" variable that is used to determine if the
test should reboot on success and not touch the no_reboot variable.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When BISECT_REVERSE and BISECT_SKIP are used together with boot or test
testing, build failures are treated as boot or test failures and
'git bisect bad' is executed instead of 'git bisect skip'. This is because
the $ret value of -1 is treated as a build failure, but the $reverse_bisect
logic does not properly handle this.
Simple fix, only invert it if it is positive.
Link: http://lkml.kernel.org/r/1335235380-8509-1-git-send-email-Russ.Dill@ti.com
Signed-off-by: Russ Dill <Russ.Dill@ti.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Pull ktest changes from Steven Rostedt.
* tag 'ktest-v3.4' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest:
ktest: Allow a test to override REBOOT_ON_SUCCESS
ktest: Fix SWITCH_TO_GOOD to also reboot the machine
ktest: Add SCP_TO_TARGET_INSTALL option
ktest: Add warning when bugs are ignored
ktest: Add INSTALL_MOD_STRIP=1 when installing modules
The option REBOOT_ON_SUCCESS is global, and will have the machine reboot
the the box if all tests are successful. But a test may not want the
machine to reboot, and perhaps have the kernel it loaded be used to
install the next kernel. Or the last test may set up a kernel that the
user may want to look at. In this case, the user could have the global
option REBOOT_ON_SUCCESS be true, but if a test is defined to run at the
end, that test can override the global option and keep the kernel it
installed for the user to log in with.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When the option SWITCH_TO_GOOD is set, it will be called when the system
needs to reboot to the good server. But currently, this keeps the reboot
from happening. The SWITCH_TO_GOOD is just a way to get to a new kernel,
it may not mean to not reboot.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Currently the option used to scp both the modules to the target as well
as the kernel image are the same (SCP_TO_TARGET). But some embedded
boards may require them to be different. The modules may need to be put
directly on the board, but the kernel image may need to go to a
tftpserver.
Add the option SCP_TO_TARGET_INSTALL that will allow the user to change
the config so that they may have the modules and image got to different
machines.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
