Overlayfs introduces some complexity with regards to what path we have
to use to shut down the scratch filesystem: it's SCRATCH_MNT for regular
filesystems, but it's OVL_BASE_SCRATCH_MNT (i.e. the lower mount of the
overlay) if overlayfs is enabled. The helper works through all that, so
we might as well use it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
So it turns out that overlayfs can't pass FS_IOC_SHUTDOWN to the lower
filesystems and so xfstests works around this by creating shutdown
helpers for the scratch fs to direct the shutdown ioctl to wherever it
needs to go to shut down the filesystem -- SCRATCH_MNT on normal
filesystems and OVL_BASE_SCRATCH_MNT when -overlay is enabled. This
means that t_open_tmpfiles cannot simply use one of the open tempfiles
to shut down the filesystem.
Commit f8f5774722 tried to "fix" this by ripping the shutdown code out,
but this made the tests useless. Fix this instead by creating a
xfstests helper to return a path that can be used to shut down the
filesystem and then pass that path to t_open_tmpfiles so that we can
shut down the filesystem when overlayfs is enabled.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Commit f8f5774722 ("generic/530: fix shutdown failure of generic/530 in
overlay") improperly clears an overlayfs test failure by shutting down
the filesystem after all the tempfiles are closed, which totally defeats
the purpose of both generic/530 and xfs/501. Revert this commit so we
can fix it properly.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
NFSv4.2 supports reflink but does not support FIBMAP nor FIEMAP.
These 4 tests about file content can pass on NFSv4.2, but filefrag
complaints :
+/mnt/testarea/scratch/test-542/file2: FIBMAP unsupported
which is breaking golden output.
Signed-off-by: Murphy Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Run fsstress, fsync every file and directory, simulate a power failure and
then verify that all files and directories exist, with the same data and
metadata they had before the power failure.
This test has found already 2 bugs in btrfs, that caused mtime and ctime of
directories not being preserved after replaying the log/journal and loss
of a directory's attributes (such a UID and GID) after replaying the log.
The patches that fix the btrfs issues are titled:
"Btrfs: fix wrong ctime and mtime of a directory after log replay"
"Btrfs: fix fsync not persisting changed attributes of a directory"
Running this test 1000 times:
- on xfs, no issues were found
- on ext4 it has resulted in about a dozen journal checksum errors (on a
5.0 kernel) that resulted in failure to mount the filesystem after the
simulated power failure with dmflakey, which produces the following
error in dmesg/syslog:
[Mon May 13 12:51:37 2019] JBD2: journal checksum error
[Mon May 13 12:51:37 2019] EXT4-fs (dm-0): error loading journal
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Testcases are recommended to use _require_scratch_shutdown()
and _scratch_shutdown() pair helper function to test and execute
shutdown.
generic/530 formmerly used _require_scratch_shutdown() to test
whether the filesystem supports shutdown or not, while executed
the shutdown action in a raw binary (src/t_open_tmpfiles) rather
than the recommended _scratch_shutdown() helper. This will cause
a "shutdown: Inappropriate ioctl for device" error message when
testing overlay filesystem.
This patch simply move the shutdown action from the raw binary
into the packaged _scratch_shutdown() helper. That is, we remove
the "shutdown" interface of t_open_tmpfiles.c and call
_scratch_shutdown() in genric/530 and xfs/501.
Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This test will test if we can still do the following operations when a
full is full:
- buffered write into unpopulated preallocated extent
- clone the untouched preallocated extent
- fsync
- no data loss if power loss happens after above fsync
Above operations should not fail, as they takes no extra data space.
Xfs passes the test, while btrfs fails at fsync and has data loss.
The fix for btrfs is:
"btrfs: Flush before reflinking any extent to prevent NOCOW write falling
back to CoW without data reservation"
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
It should return error while changing IMMUTABLE_FL and APPEND_FL if the
process has no capability CAP_LINUX_IMMUTABLE.
However, it's not true on overlayfs after kernel version v4.19 since
the process's subjective cred is overridden with ofs->creator_cred
before calling real vfs_ioctl.
The following patch for ovl fix the problem:
"ovl: check the capability before cred overridden"
Add this testcase to cover this bug.
Signed-off-by: Jiufei Xue <jiufei.xue@linux.alibaba.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
If the filesystem does not support STATX_ATTR, like NFS, setting
both attributes and attributes_mask to 0 seems the right thing to
do. attributes_mask can be 0 only if attributes is also 0.
This situation is covered by the "&" check in the next line.
Signed-off-by: Murphy Zhou <xzhou@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
If SCRATCH_LOGDEV variable is unset, the _require_logdev check fails
earlier than the following check that decides that this test should not
be run on 'btrfs'.
Signed-off-by: David Sterba <dsterba@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add some more clone range tests that missed various "wacky" combinations
of file state. Specifically, we test reflinking into and out of rainbow
ranges (a mix of real, unwritten, hole, delalloc, and shared extents),
and also we test that we can correctly handle double-inode locking no
matter what order of inodes or the filesystem's locking rules.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Bash supports file discriptor assignment in this way. So remove the hard
coded numbers. Also close this opened fd in cleanup.
Signed-off-by: Murphy Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Currently, we call repquota to report the latest quota information
after each test case. But repquota will invoke Q_SYNC on the ext4 file
system with old quota, which may be time consuming on the low speed or
busy scratch device. If we call repquota between the "overcome
softlimit" and the "overcome hardlimit" cases, the softlimit grace time
may be exceed after repquota return, and lead to test failure.
Now, we capture the following failure when the disk is busy:
pwrite: Disk quota exceeded
Touch 3+4
Touch 5+6
+touch: cannot touch 'SCRATCH_MNT/file5': Disk quota exceeded
touch: cannot touch 'SCRATCH_MNT/file6': Disk quota exceeded
Touch 5
touch: cannot touch 'SCRATCH_MNT/file5': Disk quota exceeded
This patch reset grace time before the "overcome hardlimit" case to
avoid this failure.
Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
Acked-by: Jan Kara <jack@suse.cz>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
seek_sanity_test checks for one of several SEEK_DATA/HOLE
behaviors and allows for the default behavior of filesystems,
where SEEK_HOLE always returns EOF.
This means that if filesystem has a regression in finding
holes, the sanity test won't catch it. And indeed this regression
happened in overlayfs on kernel v4.19 and went unnoticed.
To improve test coverage, add a flag -f to seek_sanity_test to
indicate that the default behavior is not acceptable.
Whitelist all filesystem types that are expected to detect holes
and use wrapper when invoking seek_sanity_test to add the -f flag
to those filesystems.
Overlayfs inherits expected behavior from base fs type.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Acked-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
A simply reproducer from Frank Sorenson:
ftruncate(fd, 65012224)
io_prep_pwrite(iocbs[0], fd, buf[0], 1048576, 63963648);
io_prep_pwrite(iocbs[1], fd, buf[1], 1048576, 65012224);
io_submit(io_ctx, 1, &iocbs[0]);
io_submit(io_ctx, 1, &iocbs[1]);
io_getevents(io_ctx, 2, 2, events, NULL)
help to find an ext4 corruption:
**************** **************** ****************
* page 1 * * page 2 * * page 3 *
**************** **************** ****************
existing 0000000000000000 0000000000000000 0000000000000000
write 1 AAAAAAAAAAAAAA AA
write 2 BBBBBBBBBBBBBB BB
result 00AAAAAAAAAAAAAA 00BBBBBBBBBBBBBB BB00000000000000
desired 00AAAAAAAAAAAAAA AABBBBBBBBBBBBBB BB00000000000000
This issue remind us we might miss unaligned AIO test for long time.
We thought fsx cover this part, but looks like it's not. So this case
trys to cover unaligned direct AIO write test on file with different
initial truncate i_size.
The following patches fix the issue on xfs and ext4.
xfs: serialize unaligned dio writes against all other dio writes
ext4: Fix data corruption caused by unaligned direct AIO
Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
When this test finishes there is no 077.full file with output from
commands. Sometimes this information is useful for post mortem so
stop deleting the file upon test completion.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This test makes sure that we can't use stale unrecovered fs metadata to
drive a DISCARD festival on a disk and thereby destroy user data by
accident.
The following patches fixed the bug on ext4, xfs and btrfs
ext4: prohibit fstrim in norecovery mode
xfs: prohibit fstrim in norecovery mode
Btrfs: do not allow trimming when a fs is mounted with the nologreplay option
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
XFS has historically had a stale data exposure window if a crash
occurs after a delalloc->physical extent conversion but before
writeback completes to the associated extent. While this should be a
rare occurrence in production environments due to typical writeback
ordering and such, it is not guaranteed in all cases until data
extents are initialized as unwritten (or otherwise zeroed) before
they are written.
Add a test that performs selective writeback ordering to reproduce
stale data exposure after a crash. Note that this test currently
fails on XFS.
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
_require_scratch doesn't actually format the scratch device with
anything, which means that tests are required to format them before
using them. Fix tests that don't do this correctly.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Certain filesystems (ext4 w/ 1k block size) can run out of space while
running this test because they have very limited xattr storage
capabilities. If we run out of space while setting an attr, don't
bother continuing the test.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Fix the unwritten extent detector in this test to ignore post-eof
allocations because those are harmless.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This test doesn't call fsync or sync to force writeback of the first 60k
of the file, which means that we could end up with a file full of
zeroes or an empty file. Since this is a regression test that looks for
stale disk contents slipping through, change the test to look for the
stale bytes.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
When test generic/131 on nfs, the 'server.out' maybe create
later than expect. Because the server is running on background,
we should ensure the 'server.out' is exist before 'cat' it.
So, let's create the server.out manually.
Signed-off-by: ZhangXiaoxu <zhangxiaoxu5@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Darrick J. Wong
Luis Henriques
Murphy Zhou
Filipe Manana
Jeffle Xu
Qu Wenruo
Jiufei Xue
David Sterba
zhangyi (F)
Amir Goldstein
Zorro Lang
Nikolay Borisov
Brian Foster
zhangxiaoxu 00467499