Targeted fuzzing tests which destroy various pieces of filesystem or
allocation group metadata; the tests look for (a) kernel detection of
corruption, (b) xfs_repair repair of said corruption, and (c)
post-repair fs usability.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Targeted fuzzing tests which destroy various pieces of file,
directory, and symlink metadata; the tests look for (a) kernel
detection of corruption, (b) e2fsck repair of said corruption, and (c)
post-repair fs usability.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Targeted fuzzing tests which destroy various pieces of filesystem or
block group metadata; the tests look for (a) kernel detection of
corruption, (b) e2fsck repair of said corruption, and (c) post-repair
fs usability.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Introduce tests for XFS and ext4 which format a filesystem, populate
it, then uses blocktrash and e2fuzz to corrupt the metadata. The FS
is remounted, modified, and unmounted. Following that, xfs_repair or
e2fsck are run until it no longer finds errors to correct, after which
the FS is mounted yet again and exercised to see if there are any
errors remaining.
The XFS test requires an xfs_db that can handle blocktrash and v5
filesystems.
The ext4 test requires metadata_csum support in e2fsprogs.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
When testing 512 block size xfs, xfs/074 fails as
QA output created by 074
+fallocate: No space left on device
Silence is golden
That's because 40051712*512=20G < 30G.
And quote from Dave:
That was sized to give AGs of a specific size, which originally
contributed to the problem being exposed. You should change the block
count specification to a size specification so the filesysetm being
made on 4k block size filesystems remains unchanged.
size = 40051712b
= 40051712 * 4096
= 164,051,812,352
= 156452m
So set the filesystem size to 156452m explicitly.
Suggested-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Per Dave Chinner's suggestion, modify generic/064 so that it won't fail
if it finds a few more extents than it expects in its test file after
inserting ranges. When 064's test file is first created, some file
systems may use more than the ideal minimum single extent to represent
it, and this can lead to a mismatch between the actual and expected
extent count after the ranges have been inserted. Ext4 file systems
mounted with delayed allocation disabled can exhibit this behavior
if a test file's blocks happen to be allocated across regions of file
system metadata.
Also, replace the open coded counting of extents and holes with a
simpler call to _count_extents(), and clarify some comments.
Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Reviewed-by: Namjae Jeon <namjae.jeon@samsung.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
xfs/042 is really an xfs_fsr test, and it only writes about 60MB of
data. however, because it is trying to write lots of data in ENOSPC
conditions, it can take a long time as XFS does flushes to maximise
space usage at ENOSPC. e.g. on a slow 1p VM:
xfs/042 426s ... 425s
It takes a long time to write a small amount of data. To avoid this
slow flushing problem, create the fragmented files with fallocate()
rather than write(), which fails much faster as there is no dirty
data to flush before retrying the allocation and failing. THis
results in:
xfs/042 425s ... 11s
A massive reduction in runtime, such that we can consider putting
xfs/042 into the quick group....
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Test that when we have a file with multiple hard links belonging to
different parent directories, if we remove one of those links, fsync the
file using one of its other links (that has a parent directory different
from the one we removed a link from), power fail and then replay the
fsync log/journal, the hard link we removed is not available anymore and
all the filesystem metadata is in a consistent state.
This test is motivated by an issue found in btrfs, where the test fails
with:
generic/107 2s ... - output mismatch (see .../results/generic/107.out.bad)
--- tests/generic/107.out 2015-08-04 09:47:46.922131256 +0100
+++ /home/fdmanana/git/hub/xfstests/results//generic/107.out.bad
@@ -1,3 +1,5 @@
QA output created by 107
Entries in testdir:
foo2
+foo3
+rmdir: failed to remove '/home/fdmanana/btrfs-tests/scratch_1/testdir': Directory not empty
...
(Run 'diff -u tests/generic/107.out .../generic/107.out.bad' to see the entire diff)
_check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent (see .../generic/107.full)
_check_dmesg: something found in dmesg (see .../generic/107.dmesg)
$ cat /home/fdmanana/git/hub/xfstests/results//generic/107.full
_check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent
*** fsck.btrfs output ***
checking extents
checking free space cache
checking fs roots
root 5 inode 257 errors 200, dir isize wrong
unresolved ref dir 257 index 3 namelen 4 name foo3 filetype 1 \
errors 5, no dir item, no inode ref
$ cat /home/fdmanana/git/hub/xfstests/results//generic/107.dmesg
(...)
[188897.707311] BTRFS info (device dm-0): failed to delete reference to \
foo3, inode 258 parent 257
[188897.711345] ------------[ cut here ]------------
[188897.713369] WARNING: CPU: 10 PID: 19452 at fs/btrfs/inode.c:3956 \
__btrfs_unlink_inode+0x182/0x35a [btrfs]()
[188897.717661] BTRFS: Transaction aborted (error -2)
(...)
[188897.747898] Call Trace:
[188897.748519] [<ffffffff8145f077>] dump_stack+0x4f/0x7b
[188897.749602] [<ffffffff81095de5>] ? console_unlock+0x356/0x3a2
[188897.750682] [<ffffffff8104b3b0>] warn_slowpath_common+0xa1/0xbb
[188897.751936] [<ffffffffa04c5d09>] ? __btrfs_unlink_inode+0x182/0x35a [btrfs]
[188897.753485] [<ffffffff8104b410>] warn_slowpath_fmt+0x46/0x48
[188897.754781] [<ffffffffa04c5d09>] __btrfs_unlink_inode+0x182/0x35a [btrfs]
[188897.756295] [<ffffffffa04c6e8f>] btrfs_unlink_inode+0x1e/0x40 [btrfs]
[188897.757692] [<ffffffffa04c6f11>] btrfs_unlink+0x60/0x9b [btrfs]
[188897.758978] [<ffffffff8116fb48>] vfs_unlink+0x9c/0xed
[188897.760151] [<ffffffff81173481>] do_unlinkat+0x12b/0x1fb
[188897.761354] [<ffffffff81253855>] ? lockdep_sys_exit_thunk+0x12/0x14
[188897.762692] [<ffffffff81174056>] SyS_unlinkat+0x29/0x2b
[188897.763741] [<ffffffff81465197>] system_call_fastpath+0x12/0x6f
[188897.764894] ---[ end trace bbfddacb7aaada8c ]---
[188897.765801] BTRFS warning (device dm-0): __btrfs_unlink_inode:3956: \
Aborting unused transaction(No such entry).
Tested against ext3/4, xfs, reiserfs and f2fs too, and all these
filesystems currently pass this test (on a 4.1 linux kernel at least).
The btrfs issue is fixed by the linux kernel patch titled:
"Btrfs: fix stale dir entries after removing a link and fsync".
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Now that generic/038 is running on my test machine, I notice how
slow it is:
generic/038 692s
11-12 minutes for a single test is way too long.
The test is creating
400,000 single block files, which can be easily parallelised and
hence run much faster than the test is currently doing.
Split the file creation up into 4 threads that create 100,000 files
each. 4 is chosen because XFS defaults to 4AGs, ext4 still has decent
speedups at 4 concurrent creates, and other filesystems aren't hurt
by excessive concurrency. The result:
generic/038 237s
on the same machine, which is roughly 3x faster and so it (just)
fast enough to to be considered acceptible.
[Eryu Guan: reduced number of files to minimum needed to reproduce
btrfs problem reliably, added $LOAD_FACTOR scaling for longer
running.]
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
After upgrading userspace on test machines to xfsprogs-4.2.0-rc1,
lots of build warnings and failures are exposed from implicit
includes that no longer exist. Hence these need fixing to allow
fstests to build correctly.
gcc also seems to have grown new stupidities around uninitialised
variables, so fix them while touching the same files.
Signed-off-by: Dave Chinner <david@fromorbit.com>
Test that an incremental send works after a file gets one of its extents
cloned/deduplicated into lower file offsets.
This is a regression test for the problem fixed by the linux kernel patch
titled:
"Btrfs: teach backref walking about backrefs with underflowed
offset values"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
In _count_extents and _count_holes, the output of 'xfs_io -c "fiemap"'
is saved in var res, but the following "echo $res" will merge the
original output into one line. e.g.
0: [0..63]: 96..159
1: [64..127]: hole
will be
0: [0..63]: 96..159 1: [64..127]: hole
so the extent count is always 0 if there's a hole.
This makes generic/046 fail occasionally. (Seems it's easier to
reproduce when the system is under some presure, e.g. with fsstress
running.)
Tested the new _count_extents and _count_holes with generic/04[3-9] and
tests all passed as expect.
Reported-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
locktest.c has already #included <errno.h>, so remove the unneeded
(and in some cases, incorrect) manual declaration of errno as an
integer. Some C libraries use a macro to provide thread-safety for
errno, so it's incorrect to try to provide a manual declaration.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Instead of the obsolete SysV S_IEXEC, S_IREAD, and S_IWRITE, use
the Posix defines of S_I[WRX]{OTH,GRP,USR}.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Test that if we remove one hard link from an inode, evict the inode,
fsync the inode, power fail and then mount the filesystem, the hard
link we removed does not exists anymore and the filesystem metadata
is in a consistent state.
This test is motivated by an issue found on btrfs, and on an unpatched
btrfs it fails with:
FSTYP -- btrfs
PLATFORM -- Linux/x86_64 debian3 4.1.0-rc6-btrfs-next-11+
MKFS_OPTIONS -- /dev/sdc
MOUNT_OPTIONS -- /dev/sdc /home/fdmanana/btrfs-tests/scratch_1
generic/098 4s ... - output mismatch (see .../generic/098.out.bad)
--- tests/generic/098.out 2015-07-23 18:01:12.616175932 +0100
+++ .../generic/098.out.bad 2015-07-23 18:04:58.924138308 +0100
@@ -1,3 +1,6 @@
QA output created by 098
Entries in testdir:
+bar
foo
+rm: cannot remove '.../testdir/foo': Stale file handle
+rmdir: failed to remove '.../scratch_1/testdir': Directory not empty
...
_check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent ...
(...)
$ cat /home/fdmanana/git/hub/xfstests/results/generic/098.full
(...)
checking fs roots
root 5 inode 258 errors 2001, no inode item, link count wrong
unresolved ref dir 257 index 0 namelen 3 name foo filetype 1 errors 6,\
no dir index, no inode ref
unresolved ref dir 257 index 3 namelen 3 name bar filetype 1 errors 5,\
no dir item, no inode ref
(...)
Tested against ext3/4, xfs, f2fs and reiserfs.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Add -d debug dump flag to ./check to directly print a test output
to stdout, instead of just saving it into a file and showing a diff
snippet.
Useful e.g. when writing a new test.
Signed-off-by: Jan Tulak <jtulak@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
When running ./check and calling a test with a name, id is enough
to find the test (names added in 03c633bf).
If the full test path is tests/xfs/123-foo-bar, then all these
invocations should work, as long as the given part of the test name
is valid and the three-digits id is here.
./check xfs/123-foo-bar
./check xfs/123-foo
./check xfs/123
Always use full test name in results.
Signed-off-by: Jan Tulak <jtulak@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
This basically does the same as in commit
90a3bfc xfs: be compatible with older mkfs.xfs which has no v5 support
which left xfs/078 behind.
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
In manual:
# ./check --help
...
-g group[,group...] include tests from these groups
-x group[,group...] exclude tests from these groups
...
Above format is not supported, because ',' is not parsed in code.
This patch fixed it.
Before patch:
# ./check -g scrub,replace
Group "scrub,replace" is empty or not defined?
After patch:
# ./check -g scrub,replace
FSTYP -- btrfs
PLATFORM -- Linux/x86_64 kerneldev 4.2.0-rc2_HEAD_c65b99f046843d2455aa231747b5a07a999a9f3d_+
MKFS_OPTIONS -- /dev/vdd
MOUNT_OPTIONS -- /dev/vdd /var/ltf/tester/scratch_mnt
...(right result)...
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
XFS had a bug which lead to spurious checksum failures during
verification of log records during recovery. This occurred when the
filesystem was mounted for recovery with a different log buffer size
(via the 'logbsize=...' mount option from when the filesystem crashed.
Create a regression test that dirties the log using one particular log
buffer size, shuts down the fs and attempts recovery using a larger log
buffer size.
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
Darrick J. Wong
Eryu Guan
Eric Whitney
Dave Chinner
Filipe Manana
Dave Chinner
Dongsheng Yang
Theodore Ts'o
Kamalesh Babulal
Omar Sandoval
Jan Tulak
Zhao Lei
Brian Foster