Thanks to Darrick J. Wong find this issue! Current splice_f generates
file offset as below:
lr = ((int64_t)random() << 32) + random();
off2 = (off64_t)(lr % maxfsize);
It generates a pseudorandom 64-bit candidate offset for the
destination file where we'll land the splice data, and then caps the
offset at maxfsize (which is 2^63- 1 on x64), which effectively means
that the data will appear at a very high file offset which creates
large (sparse) files very quickly.
That's not what we want, and some case likes shared/009 will take
forever to run md5sum on lots of huge files.
Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Overlayfs might copy up data of file on first write of file (and
not necessarily upon open of file). So call falloc file opened
with O_RDWR and after that data must have been copied up.
[Eryu: add _require_xfs_io_command "falloc" to make sure underlying
fs have fallocate(2) support]
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Instead, call _cleanup_dump explicitly from a private _cleanup.
Remove the generic cleanup bits (rm $tmp.*) from _cleanup_dump.
The only xfs/dump test that had anything other than rm $tmp.* in
_cleanup in xfs/287, but that was _scratch_unmount, which is not
needed anyway.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Format the scratch device before using it, or else xfs_db will fail,
particularly if the previous test left a corrupt fs behind.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Make sure that we still have the scratch directory after repairing our
corrupted filesystem, because repair could have nuked it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
overlay/061 is currently the only overlay test that is expected to
fail on upstream kernel.
It is a flavor of test overlay/016 with mread in stead of pread.
The ro/rw inconsistencies related to file read()/write() API were
fixed with stacked file operations in v4.19, but the ro/rw
inconsistencies related to shared mmap read/write remain to be
fixed.
The test currently checks cache coherency between mmap read and file
write(), but this sort of cache coherency is a Linux implementation
detail not a requirement of the API.
Instead of mread vs. pwrite, check consistency of mread vs. mwrite
to shared mmap, which is required by the MMAP_SHARED API.
Because we can, perform the test on shared memory that maps files
that are already close and check also that mwrite after file is
closed is persistent. This adds test coverage for future overlayfs
writeback code.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Extend test 064 to check security.capability xattr is not lost over
copy-up of a metadata only file. This requires mounting overlay with
option metacopy=on and first trigger metadata only copy-up and then
trigger data copy-up.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This is a test case for a long existing bug, caused by
over-estimated metadata space_info::bytes_may_use.
There is one proposed patch for btrfs-progs to fix it, titled:
"btrfs-progs: balance: Sync the fs before balancing metadata chunks"
The test case itself is almost the same as btrfs/181, which uses
small files to bump the reserved space to trigger the false alert.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Kernel commit 64403612b73a ("btrfs: rework
btrfs_check_space_for_delayed_refs") is introducing a regression for
btrfs balance performance.
Since that commit will cause btrfs to commit too many transactions
for nothing during balance/relocation, it will slow balance
dramatically even we only need to relocate several megabytes.
This test case will catch the problem by using super block
generation as failure criteria.
For small chunk relocated, we will commit 6 transactions for each
block group, and the test case should only have 2 block groups, it
should only commit 12 transactions.
This test case will use 120 as the threshold to detect the failure.
And in my test environment, with kernel fix btrfs committed 14
transactions. While without the fix btrfs committed 209
transactions.
So the test case should be enough to detect the regression, while still
keep the runtime small enough for failure.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Commit a514d63882c3 ("btrfs: qgroup: Commit transaction in advance
to reduce early EDQUOT") is no longer forcing transaction commit to
reclaim space, and only commits transaction asynchronously in
advance to address it.
However the criteria used in async transaction commit is not
comprehensive, thus it doesn't reclaim space automatically.
This test case will check the behavior by:
1) Falloc a large padding file
This file will take 90% of the qgroup limit
2) Sync the fs
To reflect the qgroup changes
3) Delete the file
Qgroup won't reclaim the space until transaction committed.
4) Try to write a file
If kernel not fixed, qgroup will not automatically commit transaction
to reclaim the freed space and hit EDQUOT.
This bug is going to be fixed by a patch for kernel titled:
"btrfs: qgroup: Make qgroup async transaction commit more aggressive".
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
btrfs/131 tests the free space tree, which older kernels won't have.
We shouldn't run there.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Older kernels don't support raid56. This test is still valid for
other profiles, so skip raid56 if the kernel doesn't support it.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
btrfs/125, btrfs/148, btrfs/157, and btrfs/158 test for raid56
behavior. We shouldn't run if the kernel doesn't have support for
them.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Older kernels don't have /sys/fs/btrfs. btrfs/010 will happily run
until it goes to check its work against sysfs and finds those files
don't exist. This patch introduces a require check to ensure that
the sysfs files are present before running.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The test creates a subshell that keeps running the 'cat' command
against a test file in an infinite loop, and after it kills the
subshell it unmounts the filesystem, after which point any 'cat'
subcommand that runs after or at that time will fail resulting in an
unexpected golden output:
$ ./check btrfs/081
btrfs/081 3s ... - output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/081.out.bad)
--- tests/btrfs/081.out 2018-09-16 21:30:48.501104179 +0100
+++ /home/fdmanana/git/hub/xfstests/results//btrfs/081.out.bad 2019-01-24 20:36:18.989746185 +0000
@@ -206,5 +206,6 @@
Verifying file digests after cloning
14968c092c68e32fa35e776392d14523 SCRATCH_MNT/foo
14968c092c68e32fa35e776392d14523 SCRATCH_MNT/bar
+cat: /mnt/scratch/bar: No such file or directory
Verifying target file digest after umount + mount
14968c092c68e32fa35e776392d14523 SCRATCH_MNT/bar
...
(Run 'diff -u /home/fdmanana/git/hub/xfstests/tests/btrfs/081.out /home/fdmanana/git/hub/xfstests/results//btrfs/081.out.bad' to see the entire diff)
Ran: btrfs/081
Failures: btrfs/081
Failed 1 of 1 tests
Fix that by adding a proper trap to the reader loop function so that
the subshell waits for executed 'cat' commands when it receives
SIGTERM.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Some variables inside the test's functions were used as local but
were not being declared as such. Add the local declaration for them.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
New fsstress operation breaks fs dump/restore testing which use
fsstress, e.g xfs/068.
In _create_dumpdir_stress_num, disable splice in fsstress so that we
dump exactly the same set of files and directories.
Quote Dave's comments for future reference
"
fsstress is just creating regular files differently. It has no
impact on xfsdump does except to change the number of files created
and the directory layout.
If this new functionality were creating a new type of file that
xfsdump has to handle, or adding new attributes or changing the
metadata of the existing files, then we want to make sure xfsdump is
tested against that, and so we'd be changing the golden output after
careful checking that both xfsdump and xfs_restore are working
correctly and the file count is correct.
But when all we are doing is creating normal, regular files just
with a different syscall, it makes no sense to perturb the existing
test then we have to go and validate that the new set of files being
tested is actually scanned correctly, is complete and correct. Using
a blacklist to avoid unnecessary perturbation such as in cases like
this is the right thing to do because we've had to determine if the
new functionality is a useful addition to xfsdump/restore test
coverage or not.
"
[Eryu: add Dave's comments in commit log for future reference]
Signed-off-by: Zorro Lang <zlang@redhat.com>
Suggested-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Make sure file capabilities are not lost over copy-up when file is
opened for WRITE but nothing is actually written to it.
Following commit introduced regression where if a lower file with
CAP_SETUID is opened for writing, and capability is cleared over copy up.
bd64e57586d3 ("ovl: During copy up, first copy up metadata and then data")
A later kernel patch will fix it. This test will help avoid introducing
such regressions again.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
XFS has a bug where page writeback can end up sending data to the
wrong location due to a stale, cached file mapping. Add a test to
trigger this problem by racing background writeback with a
truncate/rewrite of the final page of the file.
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Commit fb235dc06fac ("btrfs: qgroup: Move half of the qgroup
accounting time out of commit trans") could cause ABBA deadlock
between backref lookup with write lock hold (subvolume deletion) and
other read/write operations.
It's going to be fixed by "btrfs: qgroup: Don't trigger backref walk
at delayed ref insert time".
This test will generate pwrite background workload, along with
constant subvolume creation and deletion to trigger the bug.
It needs some time to generate enough files to bump the tree height
to trigger the bug.
In my test environment, with 'unsafe' cache mode for the VM, it
triggers the bug at around 70~90 seconds. So I leave the default
runtime to 120s to make sure the bug will be triggered.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
btrfs/16[123] are all seed device related test cases, make them into
'seed' group.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Eric Sandeen recently found a bug in xfs_repair that flagged extended
attribute names containing "/" as corrupt and purged them. There's
nothing in the IRIX or Linux manuals that say anything about slashes not
being allowed (and Linux certainly allows this) so let's make sure this
continues to work.
[Eryu: use $SETFATTR and _getfattr helper]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
When running xfstests generic/423 to test system call statx() on
hard link files of NFS, it fails. error message:
[!] attr 'stx_mask' differs from ref file, 7ff != e0
The values of parameter "mask" between the original file and the
reference file are different. One is STATX_ALL;
The other is STATX_ATIME | STATX_BTIME | STATX_CTIME | STATX_MTIME.
Modify the function get_reference() to pass the "mask" in, and
change STATX_ATIME | STATX_BTIME | STATX_CTIME | STATX_MTIME to
"mask".
Signed-off-by: Cui Yue <cuiyue-fnst@cn.fujitsu.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This commit tried to fix the brokennes of the kmemleak support but it
inadvertently broke the creation of the RESULT_BASE directory which lead to
problems creating check.time file. Turns out kmemleak support in xfstests has
more problems and it needs to be majorly refactor and this commit doesn't
really solve the problem. For the time being just revert to at least allow
older configuration files, which have explicitly set RESULT_BASE to work.
This reverts commit 7fc034868d.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Zorro Lang
Vivek Goyal
Amir Goldstein
Darrick J. Wong
Qu Wenruo
Jeff Mahoney
Filipe Manana
Brian Foster
Cui Yue
Nikolay Borisov