Commit Graph

444 Commits

Author SHA1 Message Date
Martin Brandenburg 85c95c7f24 fstests: add orangefs (pvfs2) support
The filesystem was renamed OrangeFS from PVFS2 recently.  The mount
-t type remains pvfs2, so I have used that name throughout despite
the kernel module being named orangefs.

We've been maintaining this as a patch for quite some time.

The easiest way to use is to setup a single server installation with
multiple filesystems orangefs and scratch, put the following in
/etc/xfsqa.config, and run ./check -pvfs2 -g quick.

TEST_DIR=/mnt
TEST_DEV=tcp://server1vm:3334/orangefs
SCRATCH_MNT=/scratch
SCRATCH_DEV=tcp://server1vm:3334/scratch

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-04-11 12:19:28 +08:00
Filipe Manana a62c85dbab common/rc: check fs supports xfs_io falloc's specific flags
For example NFS 4.2 supports fallocate but it does not support its
KEEP_SIZE flag, so we want to skip tests that use fallocate with
that flag on filesystems that don't support it.

Suggested-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-04-08 23:22:58 +08:00
Amir Goldstein e1a49994e5 fstests: _require_chattr() must get an input arg
_require_chattr() was never intended to be called without an input
argument (specifiying the required attribute to set).

However, calling it without input arguments did work and error
was silently discarded into full test output.

Fix the function to abort on missing input argument and fix the
only test that called _require_chattr() with no input argument.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-04-07 13:11:56 +08:00
Eric Biggers 98ac3be360 generic: test encryption key revocation during concurrent I/O
Add a test which revokes a keyring key while other processes are
performing I/O on an encrypted file that was "unlocked" using that key.
The crashes unpatched kernels with filesystem encryption enabled.

This bug was present in kernels v4.2 and later.  It has been fixed in
v4.11-rc4, v4.10.7, v4.9.20, and v4.4.59.

Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Richard Weinberger <richard@nod.at>
Cc: Michael Halcrow <mhalcrow@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-04-01 14:05:32 +08:00
Zorro Lang 7687a3293c common/rc: support gluster volume start with a slash
The format of glusterfs' TEST_DEV or SCRATCH_DEV is XXX:XXX or
XXX:/XXX, but xfstests can't accept the latter now. So change
the regular expression from "\w:\w" to ":/?", to accept more
glusterfs device format.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-22 17:06:23 +08:00
Zorro Lang 4cbc0a0fa8 fstests: add GlusterFS support
Add basic GlusterFS support. Neither new GlusterFS specific tests
nor related patches are included. Only support FSTYP=glusterfs or
check -glusterfs XXX.

To test on glusterfs, two gluster volumes and two mountpoint
directories are needed.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-17 12:24:01 +08:00
Harish 516a24b965 common/dmthin: Fix grep command for finding exact value
Sometimes there are multiple occurances of meta_name, pool_name,
data_name in dmsetup command which causes failures while adding in
next line. This patch greps for names at start of line.

Signed-off-by: Harish <harish@linux.vnet.ibm.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-16 18:35:23 +08:00
Goldwyn Rodrigues a913637bd7 common/rc: Use -v option with findmnt
Since btrfs puts in the subvol name in the device, we should
use -v option to not print the bind mounts or btrfs volumes.

Otherwise if a new subvolume is set as the default volume to be
mounted, as what btrfs/009 does, then next _require_scratch fails

  SCRATCH_DEV=/dev/vdc is mounted but not on SCRATCH_MNT=/scratch - aborting
  Already mounted result:
  /dev/vdc[/newvol] /scratch

[eguan: update commit log a bit, add failure example]

Fixes: 5e6892d ("common/rc: use findmnt to check mounted device")
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-16 15:26:08 +08:00
Xiao Yang 643c685027 common/config: honor NFS_MOUNT_OPTIONS in _test_mount_opts
NFS_MOUNT_OPTIONS is always not applied on TEST_DEV when it has been
defined.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-15 18:57:04 +08:00
Eric Biggers de88e28a1d common/config: don't hard-code SELinux context
If SELinux is enabled, by default xfstests mounts its filesystems
with "-o context=system_u:object_r:nfs_t:s0" so that no SELinux
xattrs get created and interfere with tests.  However, this
particular context is not guaranteed to be available because the
context names are a detail of the SELinux policy.  The SELinux
policy on Android systems, for example, does not have a context with
this name.

To fix this without having to manually override
SELINUX_MOUNT_OPTIONS, just grab the SELinux context of the root
directory.  This is arbitrary, but it should always provide a valid
context.  And if for some reason someone is actually running
xfstests in a specific SELinux context that needs files labeled with
a particular context, then they may still override
SELINUX_MOUNT_OPTIONS.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-15 17:13:45 +08:00
Eric Biggers b4d26d4052 common/quota: remove redundant SELinux detection code
SELINUX_MOUNT_OPTIONS is already set in common/config.  Setting it again
in common/quota is not necessary.  Nor is SELINUX_MOUNT_OPTIONS specific
to quota tests, so common/quota is not the right place for it.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-13 12:14:34 +08:00
Amir Goldstein 6b825e018e filter: match $TEST_* $SCRATCH_* in beginning of path string
For example, if $TEST_DIR=/mnt, only replace instances of /mnt that
are in the beginning of a path string, e.g.:

"/mnt/mntA/mntB:/mnt/mntC" => "TEST_DIR/mntA/mntB:TEST_DIR/mntC"

With this change, there is no longer a need to check the case of
$TEST_DEV being a substring of $TEST_DIR, because that would mean
that $TEST_DIR is a prefix of $TEST_DIR and that is not possible.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-10 16:54:20 +08:00
Eryu Guan 5e6892dd8b common/rc: use findmnt to check mounted device
Doing 'grep -F "$dev on "' to find the mounted device is not
always accurate, e.g.

 SCRATCH_DEV=/vda6 is mounted but not on SCRATCH_MNT=/vda6/ovl-mnt - aborting
 Already mounted result:
 /dev/vda6 on /vda6 type xfs (rw,relatime,context=system_u:object_r:nfs_t:s0,attr2,inode64,noquota)

Fix it by using findmnt command and specifying the $dev as mount
source, print the result in "$dev $mnt" format. This works for local
filesystems, network filesystems and overlayfs, avoids all kinds of
tricky and error-prone grep pattern/regex.

Also fixed the if-then-fi format in _check_mounted_on() while we're
at it.

Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-10 16:53:41 +08:00
Dmitry Monakhov f9fde7db2f report: Add xunit format report generator
xunit[1]/junit[2] are well known report formats for tests frameworks
which supported by most of test CI frameworks(such as Jenkins [3],
Bamboo [4], Avocado [5]) Basically this is just xml document which
can be easily parsed later by external tools.

EXAMPLE:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="xfstests" errors="0" skipped="1" tests="2" time="7"  hostname="alice" timestamp="2017-02-21T15:15:06"  >
	<properties>
		<property name="SECTION" value="ext4"/>
		<property name="FSTYP" value="ext4"/>
		<property name="MOUNT_OPTIONS" value="-o acl,user_xattr "/>
		<property name="HOST_OPTIONS" value="/devel/xfstests-dev.git/configs/alice.config"/>
		<property name="XFS_MKFS_OPTIONS" value="-bsize=4096"/>
		<property name="TIME_FACTOR" value="1"/>
		<property name="LOAD_FACTOR" value="1"/>
		<property name="TEST_DIR" value="/mnt/test"/>
		<property name="TEST_DEV" value="/dev/ram0"/>
		<property name="SCRATCH_DEV" value="/dev/ram1"/>
		<property name="SCRATCH_MNT" value="/mnt/scratch"/>
		<property name="OVERLAY_UPPER_DIR" value="upper"/>
		<property name="OVERLAY_LOWER_DIR" value="lower"/>
		<property name="OVERLAY_WORK_DIR" value="work"/>
	</properties>
	<testcase classname="xfstests.ext4" name="generic/001" time="5">
	</testcase>
	<testcase classname="xfstests.ext4" name="generic/010" time="1">
		<skipped message="src/dbtest not built" />
	</testcase>
</testsuite>

Footnotes:
[1] https://xunit.github.io/docs/format-xml-v2.html
[2] http://help.catchsoftware.com/display/ET/JUnit+Format
[3] https://jenkins.io
[4] https://www.atlassian.com/software/bamboo
[5] https://github.com/avocado-framework/avocado

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-07 19:53:31 +08:00
Dmitry Monakhov e11c519918 check: prepare test report generator infrastructure
Save testcase data which later may be used by report generators
- Save failure reason to $err_msg variable
- Save number of notrun tests to $n_notrun counter, similar to
  $n_try,$n_bad

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-07 19:53:12 +08:00
Gwendal Grignou d8b1dc1d2d common/config: make SELinux protection conditional
When selinux utilities are present, xfstests add options to help
circumvent selinux protection. However, on Chrome OS, it leads to
the opposite effect, when it prevents mount to succeed.

BUG=chromium:669641
TEST=xfstest test ext4/001 completes where it use to display:
common/rc: could not mount /dev/loop29 on /usr/local/autotest/tmp/xfstests_TEST

Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-07 19:41:22 +08:00
Xiao Yang de86b28517 ext4: add _require_ext4_mkfs_feature to check different features
1) We add _require_ext4_mkfs_feature to check the specified
   feature whether it is available in mkfs.ext4 or not.

2) We apply _require_ext4_mkfs_feature in ext4/003 and remove
   _require_mkfs_mkfs_bigalloc.

3) We add _require_ext4_mkfs_feature in ext4/306.  When 64bit
   feature is supported by mkfs.ext4, ext4/306 could skip.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-02 14:51:56 +08:00
Amir Goldstein 2ab614524d overlay: mount/unmount base fs before/after running tests
When TEST/SCRATCH_DEV are configured to the base fs block device,
use this information to mount base fs before running tests,
unmount it after running tests and cycle on _test_cycle_mount
along with the overlay mounts.

This helps catching overlayfs bugs related to leaking objects in
underlying (base) fs.

To preserve expected tests behavior, the semantics are:
- _scratch_mkfs mounts the base fs, cleans all files, creates
  lower/upper dirs and keeps base fs mounted
- _scratch_mount mounts base fs (if needed) and mounts overlay
- _scratch_unmount unmounts overlay and base fs

Tests that use _scratch_unmount to unmount a custom overlay mount
and expect to have access to overlay base dir, were fixed to use
explicit umount $SCRATCH_MNT instead.

The overlay test itself, does not support formatting the base fs,
so config options like MKFS_OPTIONS and FSCK_OPTIONS are ignored
on 'check -overlay'.
The config option MOUNT_OPTIONS is used to mount the base scratch fs.
The config option TEST_FS_MOUNT_OPTS is used to mount the base test fs.
The config option OVERLAY_MOUNT_OPTIONS is used to mount both test and
scratch overlay mounts.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-01 18:35:52 +08:00
Amir Goldstein bcba40e3df overlay: configure TEST/SCRATCH vars to base fs
Instead of setting the vars TEST/SCRATCH_DEV to overlay base dirs,
allow setting them to block devices to configure the base fs partition,
where overlay dirs will be created.

For example, the following config file can be used to run tests on
xfs test/scratch partitions:

 TEST_DEV=/dev/sda5
 TEST_DIR=/mnt/test
 SCRATCH_DEV=/dev/sda6
 SCRATCH_MNT=/mnt/scratch
 FSTYP=xfs

Using the same config file, but executing './check -overlay' will
use the same partitions as base fs for overlayfs directories
and set TEST_DIR/SCRATCH_MNT values to overlay mount points, i.e.:
/mnt/test/ovl-mnt and /mnt/scratch/ovl-mnt.

The base fs should be pre-formatted and mounted when starting the test.
An upcoming change is going to support mount/umount of base fs.

The new vars OVL_BASE_SCRATCH_MNT/TEST_DIR are set to point at the
overlayfs base dirs in either legacy or new config method.
Tests should always use these vars and not the legacy SCRATCH/TEST_DEV
vars when referring to overlay base dir.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-01 18:35:48 +08:00
Amir Goldstein 6f5ba68b7b overlay: rename OVERLAY_LOWER/UPPER/WORK_DIR
As preparation step for configuring test/scratch base fs
for overlay tests:

- Rename OVERLAY_LOWER/UPPER/WORK_DIR => OVL_LOWER/UPPER/WORK
  because we want to use OVL_ prefix for all base fs vars

- Prepend "ovl-" prefix to lower/upper/work path values to
  isolate the overlay test dirs when running on a base fs
  that is also used to run non overlay tests

- Make those vars values non-configurable, because lower/upper/work
  directory names are an internal test detail which should
  not concern the user and because we wish to simplify
  and document the overlay tests setup

- Add helper _filter_ovl_dirs and use it to filter output
  of test overlay/023 which matches the constant string "work"

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-01 18:35:27 +08:00
Amir Goldstein db24846eb3 fstests: allow overlay SCRATCH_DEV to be a base fs mount point
When configure SCRATCH_DEV to a mount point (and not to a directory therein)
then user will get a false positive error in scratch tests:

 $SCRATCH_DEV=/mnt/base/scratch is mounted but not on $SCRATCH_MNT=/mnt/scratch
 Already mounted result:
 /dev/sda6 on /mnt/base/scratch type xfs (rw,relatime,attr2,inode64,noquota)

This is due to the wrong `grep -F $SCRATCH_DEV` which matches the mount
point instead of the device in that mount.
Fix _check_mounted_on() to grep the pattern "$dev on " and "$dev on $mnt"
instead of just grepping for "$dev" and "$mnt" without the " on " anchor.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-01 18:35:21 +08:00
Amir Goldstein 4e965d8516 fstests: fix test and scratch filters for overlapping DEV/MNT paths
When configuring overlay base fs, TEST_DEV/DIR and SCRATCH_DEV/MNT
are derived from the base fs mount points, where *_DEV are the
path of the base fs mount point and TEST_DIR/SCRATCH_MNT are
a directory under the base fs mount point.

This means that the overlay DEV paths are prefixes of the overlay
mount points.
Fix the test and sctach filters to check if TEST_DEV/SCRATCH_DEV is
a substring of TEST_DIR/SCRATCH_MNT and try and match the longer
string first.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-01 18:35:18 +08:00
Amir Goldstein f20654ac4a fstests: canonicalize mount points on every config section
Create helper _canonicalize_mountpoint() to check and canonicalize
a mount point path.
Use helper to canonicalize TEST_DIR and SCRATCH_MNT after parse
of every config section.

This is needed for overlay base fs mount.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-01 18:35:09 +08:00
Amir Goldstein 5e33bfcf23 fstests: use _test_mount() consistently
On start of every test run and on every test, in init_rc() helper,
the test partition is verified to be mounted, or is mounted by
the helper _test_mount().

_test_mount() uses mount options $TEST_FS_MOUNT_OPTS and not
$MOUNT_OPTIONS like _scratch_mount() does.

_test_cycle_mount(), which is called by some tests uses the
_test_mount() helper as well.

Contrary to those cases, in _require_test() helper, if test
partition is not mounted, the helper _mount_or_remount_rw()
is called to mount the test partition with $MOUNT_OPTIONS.
Although this should never happen, because of the test in
init_rc(), this case is inconsistent with the rest of the code,
so it has been changed to use _test_mount() as it should.

When running tests with a multi section configuration, and
either FSTYP or MOUNT_OPTIONS change between sections, the
helper _test_unmount() is called to unmount the old test mount
and then _mount_or_remount_rw() is called to mount it again
with new FSTYP and/or MOUNT_OPTIONS.
This is again inconsistent with the rest of the code, so
was changed to use _test_mount() and instead of checking
if MOUNT_OPTIONS have changed between sections, we check if
TEST_FS_MOUNT_OPTS were changed between sections.
Otherwise, we can leave the test partition mounted.

This change is needed to support overlay base fs mount
and for multi section config files which include overlay FSTYP.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-01 18:35:05 +08:00
Amir Goldstein 64343ded7d fstests: sanity check that test partitions are not mounted elsewhere
When $TEST_DEV is mounted at a different location then $TEST_DIR,
_require_test() aborts the test with an error:
 TEST_DEV=/dev/sda5 is mounted but not on TEST_DIR=/mnt/test

There are several problems with current sanity check:
1. the output of the error is mixed into out.bad and hard to see
2. the test partition is unmounted at the end of the test regardless
   of the fact that it not pass the sanity that we have exclusivity
3. scratch partition has a similar sanity check in _require_scratch(),
   but we may not get to it, because $SCRATCH_DEV is unmounted prior
   to running the tests (which could unmount another mount point).

To solve all these problems, introduce a helper _check_mounted_on().
It checks if a device is mounted on a given mount point and optionally
checks the mounted fs type.

The sanity checks in _require_scratch() and _require_test() are
converted to use the helper and gain the check for correct fs type.

The helper is used in init_rc() to sanity check both test and scratch
partitions, before tests are run and before $SCRATCH_DEV is unmounted.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-03-01 18:34:53 +08:00