Test that when we have the no-holes feature enabled and a specific
metadata layout, if we punch a hole that starts at file offset 0 and
fsync the file, after replaying the log the hole exists.
This currently fails on btrfs but is fixed by a patch for the linux
kernel with the following subject:
"Btrfs: fix missing hole after hole punching and fsync when using NO_HOLES"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
btrfs/085 mount and works on $FLAKEY_DEV, but it's inspecting the
underlying $SCRATCH_DEV. Since writing to dm-flakey does not
invalidate page cache of $SCRATCH_DEV, the btrfs command can read a
stalled data from page cache. We should check the flakey device to
avoid such cache inconsistency.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Verify that both full and incremental send operations can issue
clone operations when a file has extents that are shared with itself
(at different offsets of course).
This currently fails on btrfs but is addressed by a patch for the
kernel titled:
"Btrfs: send, allow clone operations within the same file"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
There are some testcases use below two kind of commands to get file
size, generalize the second way as global function _get_filesize()
to simply codes.
1. ls -l $1 | $AWK_PROG '{print $5}'
2. stat -c %s $1
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The test case checks if btrfs can trim adjacent extents across
adjacent block groups boundary correctly.
The test case craft the following extents layout:
| BG1 | BG2 | BG3 |
Bytenr: X-8M X X+512M X+1G X+1G+128M
|//////|//////| | |//|
There is a long existing bug that, when btrfs is trying to trim the
range at [X+512M, X+1G+128M), it will only trim [X+512M, X+1G).
This test case is the regression test for this long existing bug.
It will verify the trimmed bytes by using loopback device backed up
by a file, and checking the bytes used by the file to determine how
many bytes are trimmed.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians no btrfs magic string.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians someother btrfs.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
I discovered a problem in btrfs where we'd end up pointing at a block we
hadn't written out yet. This is triggered by a race when two different
files on two different subvolumes fsync. This test exercises this path
with dm-log-writes, and then replays the log at every FUA to verify the
file system is still mountable and the log is replayable.
This test is to verify the fix
btrfs: fix incorrect updating of log root tree
actually fixed the problem.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add basic test to ensure btrfs conversion functionality is tested.
This test exercies conversion to all possible types of the data
portion. This is sufficient since from the POV of relocation we are
only moving blockgroups.
v5.3 and later kernel needs the following patch to pass the test
btrfs: Fix a regression which we can't convert to SINGLE profile
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add a regression test to check if btrfs can handle high devid.
The test will add and remove devices to a btrfs fs, so that the devid
will increase to uncommon but still valid values.
The regression is introduced by kernel commit ab4ba2e13346 ("btrfs:
tree-checker: Verify dev item").
The fix is titled "btrfs: tree-checker: Fix wrong check on max devid".
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
[BUG]
When btrfs/011 is executed on a fast enough system (fully memory backed
VM, with test device has unsafe cache mode), the test can fail like
this:
btrfs/011 43s ... [failed, exit status 1]- output mismatch (see /home/adam/xfstests-dev/results//btrfs/011.out.bad)
--- tests/btrfs/011.out 2019-07-22 14:13:44.643333326 +0800
+++ /home/adam/xfstests-dev/results//btrfs/011.out.bad 2019-09-18 14:49:28.308798022 +0800
@@ -1,3 +1,4 @@
QA output created by 011
*** test btrfs replace
-*** done
+failed: '/usr/bin/btrfs replace cancel /mnt/scratch'
+(see /home/adam/xfstests-dev/results//btrfs/011.full for details)
...
[CAUSE]
Looking into the full output, it shows:
...
Replace from /dev/mapper/test-scratch1 to /dev/mapper/test-scratch2
# /usr/bin/btrfs replace start -f /dev/mapper/test-scratch1 /dev/mapper/test-scratch2 /mnt/scratch
# /usr/bin/btrfs replace cancel /mnt/scratch
INFO: ioctl(DEV_REPLACE_CANCEL)"/mnt/scratch": not started
failed: '/usr/bin/btrfs replace cancel /mnt/scratch'
So this means the replace is already finished before we cancel it.
For fast system, it's very common.
[FIX]
In fill_scratch() after all the original file creations, do a timer
based direct IO write.
The extra write will take 2 * $wait_time, utilizing direct IO with 64K
block size, the write performance should be very comparable (although a
little faster) to replace performance.
So later cancel should be able to really cancel the dev-replace without
it finished too early.
Also, do extra check about the above write. If we hit ENOSPC we just
skip the test as the system is really too fast and the fs is not large
enough.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Often this test can fail on unmount because a 'btrfs subvolume snapshot'
command is still running and using the scratch the mount point:
btrfs/036 168s ... umount: /home/fdmanana/btrfs-tests/scratch_1: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)
_check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent
(see /home/fdmanana/git/hub/xfstests/results//btrfs/036.full for details)
This happens because when we kill the process running the do_snapshots()
function we only kill the subshell and don't wait for any processes it
has spawned to finish before the subshell exits. Fix this by setting a
trap in the do_snapshots() function, so that when the subshell receives
a SIGTERM signal it waits for any running 'btrfs subvolume snapshot'
to finish before exitting. We were also not waiting for the subshell
to exit after sending it the SIGTERM signal, so fix that as well by
calling the 'wait' command after calling 'kill' to send that signal.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Sometimes on fast enough test vm, btrfs/028 fails like:
btrfs/028 31s ... - output mismatch (see /home/adam/xfstests-dev/results//btrfs/028.out.bad)
--- tests/btrfs/028.out 2019-07-22 14:13:44.646666660 +0800
+++ /home/adam/xfstests-dev/results//btrfs/028.out.bad 2019-09-18 14:14:45.442131411 +0800
@@ -1,2 +1,3 @@
QA output created by 028
+/home/adam/xfstests-dev/tests/btrfs/028: line 64: kill: (2459) - No such process
Silence is golden
...
It's caused by killing already finished process.
There is no need for kill command to pollute the golden output, so
just redirect all of its stdout and stderr to null.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add a test case where falloc is called on multiple holes with qgroup
enabled.
This can cause qgroup reserved data space leak and false EDQUOT
error even we're not reaching the limit.
The fix is titled:
"btrfs: qgroup: Fix the wrong target io_tree when freeing
reserved data space"
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
We have generic dm-logwrites with fsstress test case (generic/482),
but it doesn't cover fs specific operations like btrfs snapshot
creation and deletion.
Furthermore, that test is not heavy enough to bump btrfs tree height
by its short runtime.
And finally, btrfs check doesn't consider dirty log as an error,
unlike ext*/xfs, that's to say we don't need to mount the fs to
replay the log, but just running btrfs check on the fs is enough.
So introduce a similar test case but for btrfs only.
The test case will stress btrfs by:
- Use small nodesize to bump tree height
- Create a base tree which is already high enough
- Trim tree blocks to find possible trim bugs
- Call snapshot creation and deletion along with fsstress
Also it includes certain workaround for btrfs:
- Allow _log_writes_mkfs to accept extra mkfs options
- Use no-holes feature
To avoid missing hole file extents.
Although that behavior doesn't follow the on-disk format spec, it
doesn't cause data loss. And will follow the new on-disk format spec
of no-holes feature, so it's better to workaround it.
And an optimization for btrfs only:
- Use replay-log --fsck/--check command
Since dm-log-writes records bios sequentially, there is no way to
locate certain entry unless we iterate all entries.
This is becoming a big performance penalty if we replay certain a
range, check the fs, then re-execute replay-log to replay another
range.
We need to records the previous entry location, or we need to
re-iterate all previous entries.
Thankfully, replay-log has already address it by providing --fsck and
--check command, thus we don't need to break replay-log command.
Please note, for fast storage (e.g. fast NVME or unsafe cache mode),
it's recommended to use log devices larger than 15G, or we can't
record the full log of just 30s fsstress run.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The test fails sporadically when trying to unmount the scratch filesystem
because a filefrag process is still running against a file that exists on
the scrach filesystem. This is because killing the subshell running the
fiemap_work() function does not wait for any filefrag process to complete
first. We need to set a trap for the SIGTERM signal on the subshell so
that it waits for any filefrag process before exitting.
The failure resulted in error messages like the following:
btrfs/079 57s ... umount: /home/fdmanana/btrfs-tests/scratch_1: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)
_check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent
(see /home/fdmanana/git/hub/xfstests/results//btrfs/079.full for details)
Fix this by adding a trap for SIGTERM.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
In order to check that the filesystem generation does not change
after failure to set a property, the test expects a specific
generation number of 7 in its golden output. That currently works
except when using the v2 space cache mount option (MOUNT_OPTIONS="-o
space_cache=v2"), since the filesystem generation is 8 because
creating a v2 space cache adds an additional transaction commit. So
update the test to not hardcode specific generation numbers in its
golden output and just output an unexpected message if the
generation number changes.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that an incremental send operation works after deduplicating into the
same file in both the parent and send snapshots.
This currently fails on btrfs and a kernel patch to fix it was submitted
with the subject:
Btrfs: fix incremental send failure after deduplication
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The test currently uses offsets and lengths which are multiples of
4K, but not multiples of 64K (or any other page size between 4Kb and
64Kb). This makes the reflink calls fail with -EINVAL because
reflink only operates on ranges that are aligned to the the
filesystem's block size.
Fix this by ensuring all ranges passed to the reflink calls are
aligned to 64K, so that the test works on any system regardless of
its page size. The test still fails without the corresponding
kernel fix applied [1] as it is supposed to.
[1] 3c850b45110950 ("Btrfs: incremental send, fix emission of invalid clone operations")
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Right at the beginning of the test we are unmonting the scratch device,
however at this point the device was never mounted, so the unmount fails
with an error message like the following:
umount: /home/fdmanana/btrfs-tests/scratch_1: not mounted
Which is not expected by the golden output and therefore causes the test
to fail.
Since the device/mount point was not mounted yet in the test, and since
the test framework unmounts the scratch device after each test finishes,
just remove the call to _scratch_unmount.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The current btrfs/048 test-case did not check the behavior of
properties with options like compression and with the compression
level supplied.
Add test cases for compression with compression level as well so we
can be sure we don't regress there.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
There is a long lived bug that btrfs wait for readahead to finish
indefinitely when readahead zone is inserted into seed devices.
Current write size to the file "foobar" is too small to run readahead
before the replacing on seed device. So, increase the write size to
reproduce the issue.
Following patch fixes it:
"btrfs: start readahead also in seed devices"
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
There are two regressions related to balance resume:
- Kernel NULL pointer dereference at mount time
Introduced in v5.0
- Kernel BUG_ON() just after mount
Introduced in v5.1
The kernel fixes are:
"btrfs: qgroup: Check if @bg is NULL to avoid NULL pointer
dereference"
"btrfs: reloc: Also queue orphan reloc tree for cleanup to
avoid BUG_ON()"
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that an incremental send receive does not issue clone operations
that attempt to clone the last block of a file, with a size not aligned to
the filesystem's sector size, into the middle of some other file. Such
clone request causes the receiver to fail (with EINVAL), for kernels that
include commit ac765f83f1397646 ("Btrfs: fix data corruption due to
cloning of eof block"), or cause silent data corruption for older kernels.
This test is motived by a recent regression introduced in kernel 5.2-rc1,
commit 040ee6120cb6706 ("Btrfs: send, improve clone range"), and the
following patch fixes it:
"Btrfs: incremental send, fix emission of invalid clone perations"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Filipe Manana
Naohiro Aota
Chao Yu
Qu Wenruo
Anand Jain
Josef Bacik
Nikolay Borisov
Johannes Thumshirn