Currently when no tests or test groups are specified xfstests will
silently test nothing. Interestingly enough when test groups to exclude
are specified the rest of the tests will be run.
This commit changes that to run all possible tests (for a given file
system) when no specific tests has been specified. This matches the old
xfstests behaviour.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Chandra Seetharaman <sekharan@us.ibm.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
This commit adds the possibility to specify RESULT_BASE directory from
the config file, or with environment variable. The default remains the
same "$here/results/".
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Chandra Seetharaman <sekharan@us.ibm.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
commit cbcc88fb changed test selection to use shell globs, so fixup the
examples given in the README file.
remove options for FSTYP other than NFS since the FSTYP will be automatically
detected (on Linux) from $TEST_DEV.
Signed-off-by: Dwight Engen <dwight.engen@oracle.com>
Reviewed-by: Ben Myers <bpm@sgi.com>
Signed-off-by: Ben Myers <bpm@sgi.com>
Before commit 38d58591 "xfstests: fix typo in check",
check xfs/[0-9]?? would execute all tests/xfs/[0-9]?? because:
'if grep "^$testname" $group_file >/dev/null'
returns the contents of $group_file because $testname="".
Therefore xfs/[0-9]?? was echoed to $tmp.list
Change to use egrep to fix the parsing.
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Prior to the test directory split-up, xfstests used to output the
time it previously took to run a specific test in it's output. This
was broken during the split up, as test identifiers changed and the
result output changed.
To fix this, the search for previous test results needs to look at
the sequence number rather than the absolute sequence for the test.
The new output looks the same as the old functionality:
generic/001 4s ... 5s
generic/002 1s ... 0s
generic/005 1s ... 1s
generic/006 1s ... 1s
generic/007 2s ... 1s
Where the first column is the time of the previous test run, and the
second column is the time that this run took.
Further, the check files used to generate this information are not
being output properly in the result directory, and so various log
files are not getting written to the correct location or file names.
For example, the calls to _check_test_fs would output failures to
".full", while other messages would be output to check.full, but
none would output to the corect location of RESULT_BASE/check.full.
Fix all this mess up so that all the check files for a specific run
end up in RESULT_BASE, and ensure the timing code checks the right
file and dumps output.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
After the re-factor, common.* have been renamed to common/* but there
are several files still look for the old path, fix it.
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
HOSTOS was used in a test before being initialized and this led to
failed filesystem type detection.
Signed-off-by: David Sterba <dsterba@suse.cz>
Reviewed-by: Rich Johnston <rjohnston@sgi.com>
Signed-off-by-by: Rich Johnston <rjohnston@sgi.com>
Some tests 'cd <somedir>' and then direct output to $RESULT_DIR,
which fails if the current working directory is not $here.
Regardless, if an external results directory is to be used it needs
to have a full path specified and the use of $here as the base of
the results files is completely incorrect.
Hence change all the $here/$seqres* references to simply be
$seqres*, and instead encode the full path to the results in
$RESULT_DIR.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
It is definitely handy to be able to disabdle certain tests (e.g. tests that are
known to hang or crash the test machine on certain kernels), so re-introducing
the capability of avoiding certain tests just by placing them in a file is
useful.
Introduce a command line option to specify the expunged file name. The file will
exist in each tests/* sub-directory so that only the testname is required, and
can be managed independently. The use of a command line parameter allows
multiple expunge files to exist simultaneously in the one xfstests tree which
simplifies management of a source repository used for multiple versions of a
distro.
Typical usage:
$ cat tests/generic/3.0-stable-avoid
280
$ sudo ./check -X 3.0-stable-avoid generic/280
FSTYP -- xfs (debug)
PLATFORM -- Linux/x86_64 test-1 3.0.39-dgc+
MKFS_OPTIONS -- -f -bsize=4096 /dev/vdb
MOUNT_OPTIONS -- /dev/vdb /mnt/scratch
generic/280 [expunged]
Passed all 0 tests
$
Eventually we should be able to automate setting up expunged files based on
distro release or kernel version through this infrastructure.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Introduce a top level common directory and move all the common.*
files into it. Because there is now a directory named common, the
prefix can be dropped from all the files. Convert all the tests to
use this new directory for including common files.
for f in common.*; do \
git mv `echo -n "$f " ; echo $f | sed -e 's;n\.;n/;'` \
done
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
[rjohnston@sgi.com reworked for TOT changes]
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Currently each test gets it's sequence number from it's name. It
separates this from the path via basename, and uses it for
outputting full, notrun and other status/log files. Hence these end
up in the top level directory.
All these output files need to go somewhere other than the top level
directory. Right now the check script is looking for them in the new
test directories (e.g. for the notrun files), but it would be good
to be able to separate the test source form the test output.
Hence create an output directory which has a similar heirarchy to
the test source directory. Create it on demand when we build the
list of tests to run if it doesn't already exist.
Change the high level check script to set up this variable
appropriately for each test that is being run, and to use this new
output directory for it's result files as well. The next commit will
change all the tests themselves to use the new output directory.
This is the first (small) step in being able to store test results
in an external location for archival/data mining purposes.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
[rjohnston@sgi.com rm whitespace and fix typos in commit message]
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Now that there are no tests in the top level directory, the exlude
group functionality no longer works (-x group) as it cannot find
tests to build an initial list of tests. Rework the the exclude
group implementation to operate correctly on the new test locations.
Also, switch from sed to grep for exclusion because sed has problems
distnguishing the '/' in path names from regex control...
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Now that tests are in separate subdirectories, they cannot be selected purely by
test number on the command line. We need to specify the test by subdir and test
name, so effectively we move to specifying them by regexes on the command line
rather than by expanding a range internally. This is needed to support
non-numeric test names as well, so the change may as well be made here.
This means the command line parsing needs to change from trying to detect tests
by a regex match to a processing loop that simply parses the tests and checks
for there existence. Hence the moment we find an argument that is not a switch
(i.e no preceeding "-"), we move from the arg processing loop to the test name
processing loop. IOWs, tests must be specified last on the command line.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
These are tests that are shared between multiple filesystems (moved
to shared), and udf/btrfs/ext4 specific tests, moved to appropriate
directories.
I created the "shared" directory to indicate tests that are not
truly generic, but also not filesystem specific. They might rely on
a feature that is only implmented in a few filesystems and so can't
be truly generic.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
[rjohnston@sgi.com reworked for TOT changes]
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Create a specific tests/ subdirectory to hold all the tests so they
are no longer need to be in the top level directory. This patch does
not move any tests there, however, and that will be done in
subsequent commits.
The tests/ subdir will have it's own subdirectories for different
classes of tests. Initially, there will be a per-FSTYP subdirectory
for filesytsem specific tests, and a generic directory for tests
that span multiple filesystems.
Each class will have it's own group file to indicate what groups the
tests belong to, and these will be parsed appropriately by the high
level check script to build the test list.
The change in parsing results in the test output also emitting the
path to the test as well as the name of the test, instead of just a
raw number. This allows duplicate test names in the sudirecotries to
be unambiguous when the summary is written out.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Check supports an expunge file for tests - a way of marking tests
not to run even if you specify it to run. Use is like this:
$ echo 002 > expunged
test-2:~/src/xfstests-dev$ sudo ./check 001-003
FSTYP -- xfs (debug)
PLATFORM -- Linux/x86_64 test-2 3.5.0-rc1-dgc+
MKFS_OPTIONS -- -f -bsize=4096 /dev/vdb
MOUNT_OPTIONS -- /dev/vdb /mnt/scratch
001 4s ... 4s
002 - expunged
003 0s ... 0s
Ran: 001 002 003
Not run:1
Passed all 2 tests
This is arguably useful(*), but gets in the way of splitting up the
tests into multiple directories. Remove it for now, but such
functionality should be considered for re-implementation at a later
date.
(*) e.g. having a different "expunged" file for each distro release
you have to test to avoid the tests known to fail due to fixes or
features that will never be back ported to older releases....
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
This is an old script from the auto-qa days back at SGI. It no
longer is in use or, AFAIK, ever been used for xfstests. If anyone
needs it, they can pull it back out of git, so lets remove it to
simplify check.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
The options are parsed in a messy loop of option parsing and
actions on secondary arguments. Turn it into something much neater
and easy to understand rather than a mess of temporary variables
and tortured logic...
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
[rjohnston@sgi.com fixed typos in commit message]
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
There is only one user of the common file now - check. Fold the two
into one file as the split of functionality is not necessary
anymore.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Phil White <pwhite@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
The path of the syslog writer utility 'logger' is hardcoded and not
always correct, use set_prog_path.
Signed-off-by: David Sterba <dsterba@suse.cz>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
In commit 11c1d79414 "xfstests: Change the diff output of failed
tests", the diff output of a failed test was hardcoded to 10 lines to
avoid overly long output and user can get the full output by manually
running the diff. However this is not always possible and convenient,
eg. in repeated automated tests where the required information is lost
after the test round finished. Then the caputred logs do not contain
enough informatin for analysis.
Introduce the DIFF_LENGTH env variable to tune the diff size, keeping it
10 as deafult and 0 to disable the limit.
Signed-off-by: David Sterba <dsterba@suse.cz>
Reviewed-by: Ben Myers <bpm@sgi.com>
Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Currently the entire diff is printed to the stdout in the case that the
test output template differs from the actual test run. However in some
cases the diff can be _very_ long. This commit changes it so that we
print only first 10 lines of the diff.
Also indent the diff output from the left by four spaces for better
output readability.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Ben Myers <bpm@sgi.com>
Signed-off-by: Ben Myers <bpm@sgi.com>
Lukas Czerner
Dwight Engen
Rich Johnston
Eryu Guan
Dave Chinner
David Sterba