Add a regression test for the following kernel commit:
ext4: prevent data corruption with journaling + DAX
The test passes if either we successfully compare the data between
the mmap with journaling turned on and the one with journaling
turned off, or if we fail the chattr command to turn on or off
journaling. The latter is how we prevent this issue in the kernel.
[eguan: add 'dax' group]
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
This patch does the nuts and bolts of grabbing fio results and
storing them in a database in order to check against for future
runs. This works by storing the results in resuts/fio-results.db as
a sqlite database. The src/perf directory has all the supporting
python code for parsing the fio json results, storing it in the
database, and loading previous results from the database to compare
with the current results.
This also adds a PERF_CONFIGNAME option that must be set for this to
work. Since we all have various ways we run fstests it doesn't make
sense to compare different configurations with each other (unless
specifically desired). The PERF_CONFIGNAME will allow us to
separate out results for different test run configurations to make
sure we're comparing results correctly.
Currently we only check against the last perf result. In the future
I will flesh this out to compare against the average of N number of
runs to be a little more complete, and hopefully that will allow us
to also watch latencies as well.
[eguan: add required Makefile updates]
Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
In this commit a new test case is added to test that i_size races
don't occur under dio reads/writes. We add a program in /src dir,
which has a writer to issue some append dio writes. Meanwhile it
has a reader in this test to do some dio reads. As we expect,
reader should read nothing or data with 'a'. But it might read some
data with '0'.
The bug can be reproduced by this test case [1].
1. http://patchwork.ozlabs.org/patch/311761/
This ostensibly tests commit:
9fe55eea7 Fix race when checking i_size on direct i/o read
Update by Eric Sandeen:
- update to recent xfstests
- update commit log
Update by Eryu Guan:
- add aio-dio support to the test and add 'aio' group
- add ability to test different alignments
- move test from src/ to src/aio-dio-regress/
- add .gitignore entry
- rebase against latest xfstests with various minor fixes & cleanups
- update commit log
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
This can be used to trigger an assert in the current XFS code
because it can't handle the case where there are COW extents on a
file, but none at or below the range converted by the AIO completion
handler.
Note that it doesn't trigger the assert 100% but fairly reliably.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Regression case that one can write to read-only
file in a DAX mountpoint.
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Imported Josef Bacik's code from:
https://github.com/josefbacik/log-writes.git
Specialized program for replaying a write log that was recorded by
device mapper log-writes target. The tools is used to perform
crash consistency tests, allowing to run an arbitrary check tool
(fsck) at specified checkpoints in the write log.
[Amir:]
- Add project Makefile and SOURCE files
- Document the replay-log auxiliary program
- Address review comments by Eryu Guan
Cc: Josef Bacik <jbacik@fb.com>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
When mixing buffered reads and asynchronous direct writes, it is
possible to end up with the situation where we have stale data in
the page cache while the new data is already written to disk.
This issue should be fixed by patch titled:
fs: Fix page cache inconsistency when mixing buffered and AIO DIO
Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Some tests had separate output files for IRIX and Linux. Now that the
IRIX ones have been removed, rename the Linux ones and remove the
now-unneeded .cfg files and calls to _link_out_file.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
This IRIX-specific test did some basic testing of extended attributes.
Port it to Linux; this mainly involved updating it to use the 'getfattr'
and 'setfattr' programs instead 'attr'. Note that although 'attr' is
available on Linux, it's mainly for IRIX compatibility, the man page
recommends against using it on non-XFS filesystems, and it doesn't
support listing user xattrs only. (In the last point it actually
differs from IRIX 'attr', but probably no one cares anymore.) getfattr
also sorts its output by xattr name, so its output will be the same on
all filesystems unlike 'attr -l'.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
We met a kernel assertion failure recently as below:
XFS: Assertion failed: tp->t_blk_res_used <= tp->t_blk_res, file: fs/xfs/xfs_trans.c, line: 309
Eric Sandeen digged into it and find a good reproducer.
The problem comes when the several IO vectors are copied in, and
it runs into page faults, which stops the copy before all vectors
are copied. XFS sees this as a failed/short write, and so tries
to unmap the blocks & truncate away the pages in xfs_vm_write_end.
generic_perform_write is looping, and comes back around for the
other iovecs, but the page is still there, the buffer head is still
mapped, and so a new delalloc block isn't allocated - and ends up
being allocated at writeback time, despite the fact that we "should"
have accounted for it all at delalloc write time, and trips the
assert.
Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
I'm working on a set of kernel patches to change how writeback errors
are handled and reported in the kernel. Instead of reporting a
writeback error to only the first fsync caller on the file, it has
the the kernel report them once on every file description that was
open at the time of the error.
This patch adds a test for this new behavior. Basically, open many fds
to the same file, turn on dm_error, write to each of the fds, and then
fsync them all to ensure that they all get an error back.
To do that, I'm adding a new tools/dmerror script that the C program
can use to load the error table from the script. It's also suitable for
setting up, frobbing and tearing down a dmerror device for by-hand testing.
For now, only ext2/3/4 and xfs are whitelisted on this test, since those
filesystems are included in the initial patchset. We can add to that as
we convert filesystems, and eventually make it a more general test.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Add test checking for a race in ext4 writeback that could result in
zeroing too much from the tail page during writeback.
[eguan: removed from quick group, it needs longer time on xfs and
btrfs]
Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
This adds a regression test for the following kernel patches:
mm: avoid spurious 'bad pmd' warning messages
dax: Fix race between colliding PMD & PTE entries
The above patches fix two related PMD vs PTE races in the DAX code.
These can both be easily triggered by having two threads reading and
writing simultaneously to the same private mapping, with the key
being that private mapping reads can be handled with PMDs but
private mapping writes are always handled with PTEs so that we can
COW.
Without this 2-patch kernel series, the newly added test will result
in the following errors:
run fstests generic/437 at 2017-05-16 16:53:43
mm/pgtable-generic.c:39: bad pmd ffff8808daa49b88(84000001006000a5)
... a bunch of the bad pmd messages ...
BUG: Bad rss-counter state mm:ffff8800a8c1b700 idx:1 val:1
BUG: non-zero nr_ptes on freeing mm: 38
XFS (pmem0p1): Unmounting Filesystem
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Add a test which verifies that dentries in an encrypted directory
are invalidated when an encryption key is added --- which should
cause the plaintext filenames to be visible and accessible,
replacing the encoded ciphertext filenames and any negative dentries
for the plaintext names. This primarily tests for a bug which was
fixed in the v4.5 kernel, plus a v4.6 fix for incorrect RCU usage in
the earlier fix.
Cc: linux-fscrypt@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
This adds a regression test for the following kernel patch:
dax: fix data corruption due to stale mmap reads
The above patch fixes an issue where users of DAX can suffer data
corruption from stale mmap reads via the following sequence:
- open an mmap over a 2MiB hole
- read from a 2MiB hole, faulting in a 2MiB zero page
- write to the hole with write(3p). The write succeeds but we incorrectly
leave the 2MiB zero page mapping intact.
- via the mmap, read the data that was just written. Since the zero page
mapping is still intact we read back zeroes instead of the new data.
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
- Request LIBTOOL to be used
- Set topbuildir based on a Makefile variable to call libtool
- Use /usr/local instead of /var for xfstest final location
- Move macros from aclocal.m4 to acinclude.m4, aclocal.m4 is autogenerated.
- Use autoconf variables @prefix@, @exec_prefix@.
The regular way of compiling xfstests - make - remains.
But it now runs autoreconf and libtoolize -i to produce a valid
configure.
Verified with 'make install --dry-run' that files are installed at the
same place.
Verified compiling in chromeOS chroot works as well.
[eguan: resolve merge conflicts and update .gitignore and remove
generated files by realclean]
Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
This is a clone of src/stale_handle.c test that uses generic
open_by_handle_at() syscall instead of the xfs specific ioctl.
No test is using this program yet.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Add a statx test script that does the following:
(1) Creates one each of the various types of file object and creates a
hard link to the regular file.
Note that the creation of an AF_UNIX socket is done with netcat in a
bash coprocessing thread. This might be best done with another
in-house helper to avoid a dependency on nc.
(2) Invokes the C test program included in this patch after the creation
and hands it a list of things to check appropriate to each object.
(3) Asks the test program to check the creation time of each object
against that of the preceding object.
(4) Makes various tests on the timestamps of the hardlinked file.
The patch also creates a C[*] test program to do the actual stat checking.
The test program then does the following:
(1) Compares the output of statx() to that of fstatat().
(2) Optionally compares the timestamps to see that they're sensibly
ordered with respect to each other.
(3) Optionally compares the timestamps to those of a reference file.
(4) Optionally compares the timestamps to a specified time.
(5) Optionally compares selected stats to values specified on the command
line.
(6) Optionally compares all the stats to those of a reference file,
requiring them to be the same (hard link checking).
For example:
./src/stat_test /dev/null \
stx_type=char \
stx_rdev_major=3 \
stx_rdev_minor=8 \
stx_nlink=1 \
ref=/dev/zero \
ts=B,b
The test program can also be given a --check-statx parameter to give a
quick exit code-based answer on whether statx() exists within the kernel.
[*] Note that it proved much easier to do this in C than trying to do it in
shell script and trying parsing the output of xfs_io. Using xfs_io has
other pitfalls also: it wants to *open* the file, even if the file is
not an appropriate type for this or does not grant permission to do so.
I can get around this by opening O_PATH, but then xfs_io fails to
handle XFS files because it wants to issue ioctls on every fd it opens.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Add an auxiliary program to create an AF_UNIX socket at the
specified location so that tests can do things with it.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Test if direct write invalidates pagecache correctly, so that
subsequent buffer read reads the correct data from disk.
This test is inspired by LTP tests dio29, and serves as a regression
test for the bug found by it, see kernel commit c771c14baa33
("iomap: invalidate page caches should be after iomap_dio_complete()
in direct write").
The test can be easily expanded to other write/read combinations,
e.g. buffer write + direct read and direct write + direct read, so
they are also being tested.
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
In a DAX mountpoint, do IO betwen files with and
without DAX per-inode flag. We do mmap, both
O_DIRECT and buffered read/write IO in this case.
Then test again in the same device without dax
mountoption.
Add help _require_scratch_dax to make sure we can
test DAX feature on SCRATCH_DEV.
Add mmap dio test programme to test read/write
between a mmap area of one file and another file
directly or buffered, with different size.
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
_supports_filetype() helper checks if the filetype feature
is enabled for xfs and ext* file sytems.
Add a check for the generic case where we don't know
how to test file system filetype feature.
Introduce a helper utility t_dir_type that lists directory
entries filtered by file type.
Check for filetype feature by expecting to find no directory
entries listed as DT_UNKNOWN inside a test directory.
[eguan: declare temp vars as local]
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
There have been a couple of logic bugs in `btrfs_get_extent()` which
could lead to spurious -EEXIST errors from read or write. This test
exercises those conditions by having two threads race to add an
extent to the extent map.
This is fixed by Linux commit 8dff9c853410 ("Btrfs: deal with
duplciates during extent_map insertion in btrfs_get_extent") and the
patch "Btrfs: deal with existing encompassing extent map in
btrfs_get_extent()"
(http://marc.info/?l=linux-btrfs&m=147873402311143&w=2).
Although the bug is Btrfs-specific, nothing about the test is.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Ross Zwisler
Josef Bacik
Zheng Liu
Christoph Hellwig
Xiong Zhou
Amir Goldstein
Zorro Lang
Eric Biggers
Jeff Layton
Jan Kara
Gwendal Grignou
David Howells
Eryu Guan
Omar Sandoval