Run fsstress, fsync every file and directory, simulate a power failure and
then verify that all files and directories exist, with the same data and
metadata they had before the power failure.
This test has found already 2 bugs in btrfs, that caused mtime and ctime of
directories not being preserved after replaying the log/journal and loss
of a directory's attributes (such a UID and GID) after replaying the log.
The patches that fix the btrfs issues are titled:
"Btrfs: fix wrong ctime and mtime of a directory after log replay"
"Btrfs: fix fsync not persisting changed attributes of a directory"
Running this test 1000 times:
- on xfs, no issues were found
- on ext4 it has resulted in about a dozen journal checksum errors (on a
5.0 kernel) that resulted in failure to mount the filesystem after the
simulated power failure with dmflakey, which produces the following
error in dmesg/syslog:
[Mon May 13 12:51:37 2019] JBD2: journal checksum error
[Mon May 13 12:51:37 2019] EXT4-fs (dm-0): error loading journal
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
commit 34081087 adds more sanity to seek_sanity_test() to prevent
a lseek() implementation regression from being ignored, and a
hardcoded whitelist is maintained to distinguish whether a
filesystem type only supports non-default behavior of SEEK_HOLE
or not.
In commit 34081087, NFS is listed in this whitelist, that is, NFS
is thought supporting non-default behavior only. However as far as
I know, nfsv2 and nfsv3 only support default behavior of SEEK_HOLE
(that is, always returning EOF) in linux.
On the other hand, xfstests uses "mount -t nfs ..." to mount a
NFS mount point. Normally the mount point is mounted as nfsv4,
but it can be mounted mandatorily as nfsv3 if we specify
"Nfsvers=3" in /etc/nfsmount.conf. In this case, a series of
tests fail (including generic/285, generic/448, generic/490, etc.)
with error message "Default behavior is not allowed. Aborting."
So I just make some special handling for NFS in
_fstyp_has_non_default_seek_data_hole(), that is, default behavior
of SEEK_HOLE is acceptable for nfsv2 and nfsv3.
Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Testcases are recommended to use _require_scratch_shutdown()
and _scratch_shutdown() pair helper function to test and execute
shutdown.
generic/530 formmerly used _require_scratch_shutdown() to test
whether the filesystem supports shutdown or not, while executed
the shutdown action in a raw binary (src/t_open_tmpfiles) rather
than the recommended _scratch_shutdown() helper. This will cause
a "shutdown: Inappropriate ioctl for device" error message when
testing overlay filesystem.
This patch simply move the shutdown action from the raw binary
into the packaged _scratch_shutdown() helper. That is, we remove
the "shutdown" interface of t_open_tmpfiles.c and call
_scratch_shutdown() in genric/530 and xfs/501.
Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This test requires us to fragment free space, and in part accomplishes
this by fallocating 400M of a 512M filesystem, then fallocating another
70M, and then using dd to eat remaining space. However, it's risky to
assume the 400M figure because new features such as reflink and rmap
have per-ag metadata reservations which add to overhead. Therefore,
reserve the 70M fragment file first, then try to fallocate 95% of the
remaining free space.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Tested-by: Yang Xu<xuyang2018.jy@cn.fujitsu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This test seems to check that log sizes scale up properly with the size
of the filesystem, given a carefully controlled set of mkfs parameters.
Since turning on reflink or rmap will change the minimum log size,
change the test to detect their presence and ensure they're disabled.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Tested-by: Yang Xu<xuyang2018.jy@cn.fujitsu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Create a new helper function to discover the minimum log size that will
work with the mkfs options provided, then remove all the hardcoded block
sizes from various xfs tests. This will be necessary when we turn on
reflink or rmap by default and the minimum log size increases.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Tested-by: Yang Xu<xuyang2018.jy@cn.fujitsu.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This test will test if we can still do the following operations when a
full is full:
- buffered write into unpopulated preallocated extent
- clone the untouched preallocated extent
- fsync
- no data loss if power loss happens after above fsync
Above operations should not fail, as they takes no extra data space.
Xfs passes the test, while btrfs fails at fsync and has data loss.
The fix for btrfs is:
"btrfs: Flush before reflinking any extent to prevent NOCOW write falling
back to CoW without data reservation"
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
It should return error while changing IMMUTABLE_FL and APPEND_FL if the
process has no capability CAP_LINUX_IMMUTABLE.
However, it's not true on overlayfs after kernel version v4.19 since
the process's subjective cred is overridden with ofs->creator_cred
before calling real vfs_ioctl.
The following patch for ovl fix the problem:
"ovl: check the capability before cred overridden"
Add this testcase to cover this bug.
Signed-off-by: Jiufei Xue <jiufei.xue@linux.alibaba.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Since e2fsprogs v1.44.0, debugfs with stat command shows the output:
---------------------------------------------------------------------
debugfs -R 'stat /attrfile' /dev/sda11 2> /dev/null | grep 'File ACL:'
File ACL: 9258
---------------------------------------------------------------------
Before e2fsprogs v1.44.0, debugfs with stat command shows the output:
----------------------------------------------------------------------
debugfs -R 'stat /attrfile' /dev/sda11 2> /dev/null | grep 'File ACL:'
File ACL: 9258 Directory ACL: 0
----------------------------------------------------------------------
"Directory ACL: XXXX" was removed by commit 578fcbf so running ext4/018
with older e2fsprogs(i.e. before v1.44.0) got the following error:
--------------------------------------------------------------------------
+./tests/ext4/018: line 59: 9258 Directory ACL: 0: syntax error in expression (error token is "Directory ACL: 0")
--------------------------------------------------------------------------
we match the value of File ACL accurately to fix it.
Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
If the filesystem does not support STATX_ATTR, like NFS, setting
both attributes and attributes_mask to 0 seems the right thing to
do. attributes_mask can be 0 only if attributes is also 0.
This situation is covered by the "&" check in the next line.
Signed-off-by: Murphy Zhou <xzhou@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
If SCRATCH_LOGDEV variable is unset, the _require_logdev check fails
earlier than the following check that decides that this test should not
be run on 'btrfs'.
Signed-off-by: David Sterba <dsterba@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Remove the typedef usage for the xfs geometry structure, which will
be removed in future patch.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The recent commit 4529b20e1a ("btrfs/048: amend property validation
cases"), does not properly filter the scratch device because the error
messages are sent to stderr and not to stdout, and the pipe filter only
gets input from the stdout of the btrfs utility. We need to redirect the
stderr of the btrfs utility to its stdout.
Further, the golden output had the path "/mnt/scratch" hardcoded, instead
of using SCRATCH_MNT. Fix that as well.
The test was failing on any setup where the scratch device is not mounted
at "/mnt/scratch".
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Stress send running in parallel with balance and deduplication against
files that belong to the snapshots used by send. The goal is to verify
that these operations running in parallel do not lead to send crashing
(trigger assertion failures and BUG_ONs), or send finding an inconsistent
snapshot that leads to a failure (reported in dmesg/syslog). The test
needs big trees (snapshots) with large differences between the parent and
send snapshots in order to hit such issues with a good probability.
This currently fails on btrfs, hitting a BUG_ON() often, and with btrfs
error messages in dmesg/syslog. The problem has always existed and it is
not new, but probably unnoticed due to lack of test cases that exercise
these btrfs features running in parallel.
The following patches for btrfs fix the problems:
"Btrfs: fix race between send and deduplication that lead to failures and
crashes"
"Btrfs: prevent send failures and crashes due to concurrent relocation"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Ignore lockdep complaining about its own bugginess when scanning dmesg
output, because we shouldn't be failing filesystem tests on account of
lockdep.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Wipe the scratch devices in between each test to ensure that tests are
formatting them and not making assumptions about previous contents.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Remove the require_{test,scratch]* sentinel files after a test fails.
This eliminates false fsck corruption reports such as the following:
1. Test A calls _require_scratch, which creates the sentinel file
$RESULT_DIR/require_scratch to facilitate fsck after the test completes.
2. Test A runs some test, which corrupts the scratch filesystem due to
kernel bug or something.
3. Test A calls _fail because of the errors in (2). Note that the test
case returned 1, so ./check unmounts the test and scratch filesystems
without checking them or removing $RESULT_DIR/require_scratch
4. Test B starts up, but does not call _require_scratch. The
$RESULT_DIR/require_scratch file is still there.
5. Test B completes successfully.
6. ./check calls _check_filesystems, which sees the
$RESULT_DIR/require_scratch file and runs fsck.
7. fsck reports the corrupt scratch device (which is associated with
test B) even though B did not ever touch the scratch device and it was
actually test A that corrupted the filesystem.
Note that with the "check: wipe scratch devices between tests" patch
applied, we can also reproduce this problem by running xfs/172 and
xfs/195 with a scratch device small enough that the files created in 172
span multiple AGs and therefore cause 172 to fail.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that if we have a subvolume/snapshot that is writable, has a file
with unflushed delalloc (buffered writes not yet flushed), turn the
subvolume to readonly mode and then use it for send a operation, the send
stream will contain the delalloc data - that is, no data loss happens.
This currently files on btrfs (data loss) but is fixed by a patch for
the linux kernel titled:
"Btrfs: send, flush dellaloc in order to avoid data loss"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Currently fssum, mostly used for btrfs test cases that test the btrfs send
feature, ignores completely the existence of xattrs. This change teaches
fssum to find xattrs and make them contribute to the checksum of a
filesystem, so that we can catch filesystem bugs regarding missing, corrupt
or not supposed to exist xattrs (i.e. that an incremental btrfs send does
not forget to create, update or remove xattrs).
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The previous patches added support for operations to set, get and delete
xattrs on regular files and directories, this patch just adds an operation
to list the xattrs of a file/directory.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The previous patches added support for operations to set and get xattrs on
regular files and directories, this patch just adds one operation to delete
xattrs on files and directories.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The previous patch added support for an operation to set xattrs on regular
files and directories, this patch just adds one operation to read (get)
them.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Currently fsstress does not exercise creating, reading or deleting xattrs
on files or directories. This change adds support for setting xattrs on
files and directories, using only the xattr user namespace (the other
namespaces are not general purpose and are used for security, capabilities,
ACLs, etc). This adds a counter for each file entry structure that keeps
track of the number of xattrs set for the file entry, and each new xattr
has a name that includes the counter's value (example: "user.x4").
Values for the xattrs have at most 100 bytes, which is more than the
maximum size supported for all major filesystems.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Currently the afsync function can only be performed against regular files.
Allow it to operate on directories too, to increase test coverage and
allow for chances of finding bugs in a filesystem's implementation of
fsync against directories.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Currently the fsync function can only be performed against regular files.
Allow it to operate on directories too, to increase test coverage and
allow for chances of finding bugs in a filesystem's implementation of
fsync against directories.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Filipe Manana
Jeffle Xu
Darrick J. Wong
Qu Wenruo
Jiufei Xue
Xiao Yang
Murphy Zhou
David Sterba