Commit 52d21580b3 "ktest: Use make -s kernelrelease" fixed commit
7ff525712a "kbuild: fake the "Entering directory ..." message more simply"
as that commit added output after the make kernelrelease. But there's still
some build scripts that are used by ktest that has output before the make
is executed, and requires that only the last line is printed.
Cc: Michal Marek <mmarek@suse.cz>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Instead of just showing the test type of test in the start of the
test, like this:
RUNNING TEST 1 of 26 with option build defconfig
Add the name (if it is defined) as well, like this:
RUNNING TEST 1 of 26 (arm64 aarch64-linux) with option build defconfig
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Tests can set options that override the default ones. But if a test
tries to undefine a default option, it is simply ignored and the
default option stays as is.
For example, if you want to have a test that defines no MIN_CONFIG
then the test should be able to do that with:
TEST_START
MIN_CONFIG =
Which should make MIN_CONFIG not defined for that test. But the way
the code currently works, undefined options in tests are dropped.
This is because the NULL options are evaluated during the reading of
the config file and since one can disable default options in the default
section with this method, it is evaluated there (the option turns to a
undef). But undef options in the test section mean to use the default
option.
To fix this, keep the empty string in the option during the reading
of the config file, and then evaluate it when running the test. This
will allow tests to null out default options.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Commit 6071c22e17 "ktest: Rewrite the config-bisect to actually work"
fixed the config-bisect to work nicely but in doing so it broke
make_min_config by changing the way assign_configs works.
The assign_configs function now adds the config to the hash even if
it is disabled, but changes the hash value to be that of the
line "# CONFIG_FOO is not set". Unfortunately, the make_min_config
test only checks to see if the config is removed. It now needs to
check if the config is in the hash and not set to be disabled.
Cc: stable@vger.kernel.org # 3.17+
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If git bisect reply is being used in the bisect tests, don't bother
doing the git bisect good or git bisect bad calls. The git bisect
reply will override them anyway, and that's called immediately
after the other two. Going the git bisect (good|bad) is just a
waste of time.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The reboot function when rebooting back to a good kernel has a check
to make sure that a new kernel was indeed booted. But that check
uses a timeout value, which when calling the monitor will still
return success if the timeout is hit (no bug was found). It should
return an error to let the reboot code know that a new kernel was
not reached. Only the reboot code checks the return value of the
monitor.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add a way to run a patchcheck test on the commits that are in one branch
but not in another. This uses git cherry to find a list of commits to
test each one with.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
With the more robust config_bisect, the documentation is out of
date and needs to be updated.
The new rewrite allows for finding missing configs and such, and
is much more robust to use.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
After the rewrite of the config bisect, there were several unused
functions that can be removed.
One of the unused functions printed out the failed config nicer than
what the rewrite did, so I kept that and used it to output the
bad config.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The new rewrite left out the CONFIG_BISECT_CHECK, which allows the
user to test that their "bad" config still is bad and their "good"
config still is good. This is especially important as the configs
are passed through a "make oldconfig" to update them with the lastest
kernel. Things could change that causes a bad config to work, or a
good config to break. The check is done after the configs have run
through the oldconfig processing.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
I never liked the way config-bisect worked. I would assume the bad config
had some config that broke the system. But it would not work if the bad
config just happened to be missing something that the good config had.
I rewrote the config-bisect to do this properly. It does a diff of the two
configs, and sets half of the configs that are in one and not the other.
The way it works is that when it "sets", it really just makes one copy
what the other has. That is, a "set" can be setting a:
# CONFIG_FOO is not set
Basically, it looks at the differences between the two files and makes
them similar until it comes down to one config that makes it work or
not work depending on if it is set or not.
Note, if more than one config change makes the bad config not work, it
will only find one of them. But this is true with all bisect logic.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Some cleanup for improving readability as follows.
- Initialize $ktest_config at its definition.
- Put parentheses around the `config-file' argument in the usage message
because it's a optional one.
- Rename get_ktest_config{,s} to more descriptive get_mandatory_config{,s}.
Link: http://lkml.kernel.org/r/87fvmr30kb.wl%satoru.takeuchi@gmail.com
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Pull ktest updates from Steven Rostedt:
"Here's some basic updates to ktest.pl. They include:
- add config to modify the signal to terminate console
- update to documentation (missing some config options)
- add KERNEL_VERSION variable to use for other configs
- add '=~' to let configs eval other configs
- add BISECT_TRIES to run multiple tests per git bisect good"
* tag 'ktest-v3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest:
ktest: Add BISECT_TRIES to bisect test
ktest: Add eval '=~' command to modify variables in config file
ktest: Add special variable ${KERNEL_VERSION}
ktest: Add documentation of CLOSE_CONSOLE_SIGNAL
ktest: Make the signal to terminate the console configurable
For those cases that it takes several tries to hit a bug, it would be
useful for ktest.pl to try a test multiple times before it considers
the test as a pass. To accomplish this, BISECT_TRIES ktest config
option has been added. It is default to one, as most of the time a
bisect only needs to try a test once. But the user can now up this
to make ktest run a given test multiple times. The first failure
that is detected will set a bisect bad. It only repeats on success.
Note, as with all race bugs, there's no guarantee that if it succeeds,
it is really a good bisect. But it helps in case the bug is somewhat
reliable.
You can set BISECT_TRIES to zero, and all tests will be considered
good, unless you also set BISECT_MANUAL.
Suggested-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
With the added variable ${KERNEL_VERSION}, it is useful to be
able to use parts of it for other variables.
For example, if you want to create a warnings file for each major
kernel version to test sub versions against you can create
your warnings file with like this:
WARNINGS_FILE = warnings-file-${KERNEL_VERSION}
But this may add 3.8.12 or something, and we want all 3.8.* to
use the same file, and 3.10.* to use another file, and so on.
With the eval command we can, by adding:
WARNINGS_FILE =~ s/(-file-\d+\.\d+).*/$1/
Which will chop off the extra characters after the 3.8.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Add a special variable that can be used in other variables called
${KERNEL_VERSION}. This will embed the current kernel version into
the variable. For example:
WARNINGS_FILE = ${OUTPUT_DIR}/warnings-${KERNEL_VERSION}
If the current version is v3.8 then the WARNINGS_FILE will become
${OUTPUT_DIR}/warnings-v3.8
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The sample.conf file needs to document all available options.
With the new CLOSE_CONSOE_SIGNAL option, it too needs to be
document.
Cc: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Currently ktest sends SIGINT to terminate the console.
However, there are consoles which do not exit by this signal, for example,
in my case, "virsh console <guest OS>". In such case, ktest is blocked in
close_console(). It prevents this automate test.
This patch adds new option CLOSE_CONSOLE_SIGNAL which mean the
signal to terminate the console. Since its default value is "INT",
the original behavior isn't changed.
Link: http://lkml.kernel.org/r/87zjol8pl5.wl%satoru.takeuchi@gmail.com
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
