[BUG]
With btrfs-progs v5.4, btrfs/140 and btrfs/141 will fail.
[CAUSE]
Both tests are testing re-silvering of RAID1, thus they need to corrupt
on-disk data.
This requires to do manual logical -> physical bytes mapping in the test
case.
However the test case itself uses too many hard coded helper to grab
physical offset, which will change with mkfs.btrfs.
[FIX]
Use more flex helper, to get both devid and physical for such
corruption.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
[BUG]
When using btrfs-progs v5.4, btrfs/157 and btrfs/158 will fail:
btrfs/157 1s ... - output mismatch (see xfstests/results//btrfs/157.out.bad)
--- tests/btrfs/157.out 2018-09-16 21:30:48.505104287 +0100
+++ xfstests/results//btrfs/157.out.bad
2019-12-10 15:35:43.112390076 +0000
@@ -1,9 +1,9 @@
QA output created by 157
wrote 131072/131072 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
+wrote 65536/65536 bytes at offset 22020096
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
...
(Run 'diff -u xfstests/tests/btrfs/157.out xfstests/results//btrfs/157.out.bad' to see the entire diff)
btrfs/158 2s ... - output mismatch (see xfstests/results//btrfs/158.out.bad)
--- tests/btrfs/158.out 2018-09-16 21:30:48.505104287 +0100
+++ xfstests/results//btrfs/158.out.bad
2019-12-10 15:35:44.844388521 +0000
@@ -1,9 +1,9 @@
QA output created by 158
wrote 131072/131072 bytes at offset 0
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
+wrote 65536/65536 bytes at offset 22020096
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
...
(Run 'diff -u xfstests/tests/btrfs/158.out xfstests/results//btrfs/158.out.bad' to see the entire diff)
[CAUSE]
This two tests use physical offset as golden output, while mkfs.btrfs
can do whatever it likes to arrange its chunk layout, thus physical
offset is never reliable.
And btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
order to the stripe index") just changed the layout.
So the output mismatch and failed.
[FIX]
In fact, that btrfs-progs commit not only changed offset, but also the
device sequence.
So we can't just simply remove the physical offset, but also need to use
proper helper to get both devid (as its device path) and physical offset
for corruption.
As long as mkfs.btrfs still uses sequential devid, these tests should
handle future chunk layout change without problem.
Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
[BUG]
When using btrfs-progs v5.4, btrfs/142 and btrfs/143 will fail:
btrfs/142 1s ... - output mismatch (see xfstests/results//btrfs/142.out.bad)
--- tests/btrfs/142.out 2018-09-16 21:30:48.505104287 +0100
+++ xfstests/results//btrfs/142.out.bad
2019-12-10 15:35:40.280392626 +0000
@@ -3,37 +3,37 @@
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 65536/65536 bytes
XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
-XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
-XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
-XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
...
(Run 'diff -u xfstests/tests/btrfs/142.out xfstests/results//btrfs/142.out.bad' to see the entire diff)
[CAUSE]
Btrfs/14[23] test whether a read on corrupted stripe will re-silver
itself.
Such test by its nature will need to modify on-disk data, thus need to
get the btrfs logical -> physical mapping, which is done by near
hard-coded lookup function, which rely on certain stripe:devid sequence.
Recent btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
order to the stripe index") changes how we use devices in mkfs.btrfs,
this caused a change in chunk layout, and break the hard-coded
stripe:devid sequence.
[FIX]
This patch will do full devid and physical offset lookup, instead of old
physical offset only lookup.
The only assumption made is, mkfs.btrfs assigns devid sequentially for
its devices.
Which means, for "mkfs.btrfs $dev1 $dev2 $dev3", we get devid 1 for $dev1,
devid 2 for $dev2, and so on.
This change will allow btrfs/14[23] to handle even future chunk layout
change. (Although I hope this will never happen again).
This also addes extra debug output (although less than 10 lines) into
$seqres.full, just in case when layout changes and current lookup can't
handle it, developer can still pindown the problem easily.
Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
In my testing on 1GB zram devices btrfs/187 usually fails with
ENOSPC.
Add a requirement for 8GB scratch devices (empirically measured).
Cc: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that when we have the no-holes feature enabled and a specific
metadata layout, if we punch a hole that starts at file offset 0 and
fsync the file, after replaying the log the hole exists.
This currently fails on btrfs but is fixed by a patch for the linux
kernel with the following subject:
"Btrfs: fix missing hole after hole punching and fsync when using NO_HOLES"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
btrfs/085 mount and works on $FLAKEY_DEV, but it's inspecting the
underlying $SCRATCH_DEV. Since writing to dm-flakey does not
invalidate page cache of $SCRATCH_DEV, the btrfs command can read a
stalled data from page cache. We should check the flakey device to
avoid such cache inconsistency.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Verify that both full and incremental send operations can issue
clone operations when a file has extents that are shared with itself
(at different offsets of course).
This currently fails on btrfs but is addressed by a patch for the
kernel titled:
"Btrfs: send, allow clone operations within the same file"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
There are some testcases use below two kind of commands to get file
size, generalize the second way as global function _get_filesize()
to simply codes.
1. ls -l $1 | $AWK_PROG '{print $5}'
2. stat -c %s $1
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The test case checks if btrfs can trim adjacent extents across
adjacent block groups boundary correctly.
The test case craft the following extents layout:
| BG1 | BG2 | BG3 |
Bytenr: X-8M X X+512M X+1G X+1G+128M
|//////|//////| | |//|
There is a long existing bug that, when btrfs is trying to trim the
range at [X+512M, X+1G+128M), it will only trim [X+512M, X+1G).
This test case is the regression test for this long existing bug.
It will verify the trimmed bytes by using loopback device backed up
by a file, and checking the bytes used by the file to determine how
many bytes are trimmed.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians no btrfs magic string.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians someother btrfs.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
I discovered a problem in btrfs where we'd end up pointing at a block we
hadn't written out yet. This is triggered by a race when two different
files on two different subvolumes fsync. This test exercises this path
with dm-log-writes, and then replays the log at every FUA to verify the
file system is still mountable and the log is replayable.
This test is to verify the fix
btrfs: fix incorrect updating of log root tree
actually fixed the problem.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add basic test to ensure btrfs conversion functionality is tested.
This test exercies conversion to all possible types of the data
portion. This is sufficient since from the POV of relocation we are
only moving blockgroups.
v5.3 and later kernel needs the following patch to pass the test
btrfs: Fix a regression which we can't convert to SINGLE profile
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add a regression test to check if btrfs can handle high devid.
The test will add and remove devices to a btrfs fs, so that the devid
will increase to uncommon but still valid values.
The regression is introduced by kernel commit ab4ba2e13346 ("btrfs:
tree-checker: Verify dev item").
The fix is titled "btrfs: tree-checker: Fix wrong check on max devid".
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
[BUG]
When btrfs/011 is executed on a fast enough system (fully memory backed
VM, with test device has unsafe cache mode), the test can fail like
this:
btrfs/011 43s ... [failed, exit status 1]- output mismatch (see /home/adam/xfstests-dev/results//btrfs/011.out.bad)
--- tests/btrfs/011.out 2019-07-22 14:13:44.643333326 +0800
+++ /home/adam/xfstests-dev/results//btrfs/011.out.bad 2019-09-18 14:49:28.308798022 +0800
@@ -1,3 +1,4 @@
QA output created by 011
*** test btrfs replace
-*** done
+failed: '/usr/bin/btrfs replace cancel /mnt/scratch'
+(see /home/adam/xfstests-dev/results//btrfs/011.full for details)
...
[CAUSE]
Looking into the full output, it shows:
...
Replace from /dev/mapper/test-scratch1 to /dev/mapper/test-scratch2
# /usr/bin/btrfs replace start -f /dev/mapper/test-scratch1 /dev/mapper/test-scratch2 /mnt/scratch
# /usr/bin/btrfs replace cancel /mnt/scratch
INFO: ioctl(DEV_REPLACE_CANCEL)"/mnt/scratch": not started
failed: '/usr/bin/btrfs replace cancel /mnt/scratch'
So this means the replace is already finished before we cancel it.
For fast system, it's very common.
[FIX]
In fill_scratch() after all the original file creations, do a timer
based direct IO write.
The extra write will take 2 * $wait_time, utilizing direct IO with 64K
block size, the write performance should be very comparable (although a
little faster) to replace performance.
So later cancel should be able to really cancel the dev-replace without
it finished too early.
Also, do extra check about the above write. If we hit ENOSPC we just
skip the test as the system is really too fast and the fs is not large
enough.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Often this test can fail on unmount because a 'btrfs subvolume snapshot'
command is still running and using the scratch the mount point:
btrfs/036 168s ... umount: /home/fdmanana/btrfs-tests/scratch_1: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)
_check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent
(see /home/fdmanana/git/hub/xfstests/results//btrfs/036.full for details)
This happens because when we kill the process running the do_snapshots()
function we only kill the subshell and don't wait for any processes it
has spawned to finish before the subshell exits. Fix this by setting a
trap in the do_snapshots() function, so that when the subshell receives
a SIGTERM signal it waits for any running 'btrfs subvolume snapshot'
to finish before exitting. We were also not waiting for the subshell
to exit after sending it the SIGTERM signal, so fix that as well by
calling the 'wait' command after calling 'kill' to send that signal.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Sometimes on fast enough test vm, btrfs/028 fails like:
btrfs/028 31s ... - output mismatch (see /home/adam/xfstests-dev/results//btrfs/028.out.bad)
--- tests/btrfs/028.out 2019-07-22 14:13:44.646666660 +0800
+++ /home/adam/xfstests-dev/results//btrfs/028.out.bad 2019-09-18 14:14:45.442131411 +0800
@@ -1,2 +1,3 @@
QA output created by 028
+/home/adam/xfstests-dev/tests/btrfs/028: line 64: kill: (2459) - No such process
Silence is golden
...
It's caused by killing already finished process.
There is no need for kill command to pollute the golden output, so
just redirect all of its stdout and stderr to null.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add a test case where falloc is called on multiple holes with qgroup
enabled.
This can cause qgroup reserved data space leak and false EDQUOT
error even we're not reaching the limit.
The fix is titled:
"btrfs: qgroup: Fix the wrong target io_tree when freeing
reserved data space"
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
We have generic dm-logwrites with fsstress test case (generic/482),
but it doesn't cover fs specific operations like btrfs snapshot
creation and deletion.
Furthermore, that test is not heavy enough to bump btrfs tree height
by its short runtime.
And finally, btrfs check doesn't consider dirty log as an error,
unlike ext*/xfs, that's to say we don't need to mount the fs to
replay the log, but just running btrfs check on the fs is enough.
So introduce a similar test case but for btrfs only.
The test case will stress btrfs by:
- Use small nodesize to bump tree height
- Create a base tree which is already high enough
- Trim tree blocks to find possible trim bugs
- Call snapshot creation and deletion along with fsstress
Also it includes certain workaround for btrfs:
- Allow _log_writes_mkfs to accept extra mkfs options
- Use no-holes feature
To avoid missing hole file extents.
Although that behavior doesn't follow the on-disk format spec, it
doesn't cause data loss. And will follow the new on-disk format spec
of no-holes feature, so it's better to workaround it.
And an optimization for btrfs only:
- Use replay-log --fsck/--check command
Since dm-log-writes records bios sequentially, there is no way to
locate certain entry unless we iterate all entries.
This is becoming a big performance penalty if we replay certain a
range, check the fs, then re-execute replay-log to replay another
range.
We need to records the previous entry location, or we need to
re-iterate all previous entries.
Thankfully, replay-log has already address it by providing --fsck and
--check command, thus we don't need to break replay-log command.
Please note, for fast storage (e.g. fast NVME or unsafe cache mode),
it's recommended to use log devices larger than 15G, or we can't
record the full log of just 30s fsstress run.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The test fails sporadically when trying to unmount the scratch filesystem
because a filefrag process is still running against a file that exists on
the scrach filesystem. This is because killing the subshell running the
fiemap_work() function does not wait for any filefrag process to complete
first. We need to set a trap for the SIGTERM signal on the subshell so
that it waits for any filefrag process before exitting.
The failure resulted in error messages like the following:
btrfs/079 57s ... umount: /home/fdmanana/btrfs-tests/scratch_1: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)
_check_btrfs_filesystem: filesystem on /dev/sdc is inconsistent
(see /home/fdmanana/git/hub/xfstests/results//btrfs/079.full for details)
Fix this by adding a trap for SIGTERM.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
In order to check that the filesystem generation does not change
after failure to set a property, the test expects a specific
generation number of 7 in its golden output. That currently works
except when using the v2 space cache mount option (MOUNT_OPTIONS="-o
space_cache=v2"), since the filesystem generation is 8 because
creating a v2 space cache adds an additional transaction commit. So
update the test to not hardcode specific generation numbers in its
golden output and just output an unexpected message if the
generation number changes.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that an incremental send operation works after deduplicating into the
same file in both the parent and send snapshots.
This currently fails on btrfs and a kernel patch to fix it was submitted
with the subject:
Btrfs: fix incremental send failure after deduplication
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The test currently uses offsets and lengths which are multiples of
4K, but not multiples of 64K (or any other page size between 4Kb and
64Kb). This makes the reflink calls fail with -EINVAL because
reflink only operates on ranges that are aligned to the the
filesystem's block size.
Fix this by ensuring all ranges passed to the reflink calls are
aligned to 64K, so that the test works on any system regardless of
its page size. The test still fails without the corresponding
kernel fix applied [1] as it is supposed to.
[1] 3c850b45110950 ("Btrfs: incremental send, fix emission of invalid clone operations")
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Right at the beginning of the test we are unmonting the scratch device,
however at this point the device was never mounted, so the unmount fails
with an error message like the following:
umount: /home/fdmanana/btrfs-tests/scratch_1: not mounted
Which is not expected by the golden output and therefore causes the test
to fail.
Since the device/mount point was not mounted yet in the test, and since
the test framework unmounts the scratch device after each test finishes,
just remove the call to _scratch_unmount.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The current btrfs/048 test-case did not check the behavior of
properties with options like compression and with the compression
level supplied.
Add test cases for compression with compression level as well so we
can be sure we don't regress there.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Qu Wenruo
Johannes Thumshirn
Filipe Manana
Naohiro Aota
Chao Yu
Anand Jain
Josef Bacik
Nikolay Borisov