Test that an incremental send operation works correctly when a file has
shared extents with itself in the send snapshot, with a hole between them,
and the file size has increased in the send snapshot.
This currently fails in 5.5-rc kernels (regression) but is fixed by a
patch that has the following subject:
Btrfs: send, fix emission of invalid clone operations within the same file
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
As of the linux kernel commit fd0ddbe2509568 ("Btrfs: send, skip
backreference walking for extents with many references"), the test is no
longer dangerous and it's fast (takes 1 to 2 seconds on a modest vm with
a debug kernel). Therefore remove it from the dangerous group and add it
to the auto and quick groups as well.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test removal of a subvolume via rmdir after it has been renamed into a
snapshot of the volume that originally contained the subvolume
reference.
This currently fails on btrfs but is fixed by the patch with the title
"btrfs: fix invalid removal of root ref"
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that when we have the no-holes feature enabled and a specific
metadata layout, if we punch a hole that starts at file offset 0 and
fsync the file, after replaying the log the hole exists.
This currently fails on btrfs but is fixed by a patch for the linux
kernel with the following subject:
"Btrfs: fix missing hole after hole punching and fsync when using NO_HOLES"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Verify that both full and incremental send operations can issue
clone operations when a file has extents that are shared with itself
(at different offsets of course).
This currently fails on btrfs but is addressed by a patch for the
kernel titled:
"Btrfs: send, allow clone operations within the same file"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
The test case checks if btrfs can trim adjacent extents across
adjacent block groups boundary correctly.
The test case craft the following extents layout:
| BG1 | BG2 | BG3 |
Bytenr: X-8M X X+512M X+1G X+1G+128M
|//////|//////| | |//|
There is a long existing bug that, when btrfs is trying to trim the
range at [X+512M, X+1G+128M), it will only trim [X+512M, X+1G).
This test case is the regression test for this long existing bug.
It will verify the trimmed bytes by using loopback device backed up
by a file, and checking the bytes used by the file to determine how
many bytes are trimmed.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians no btrfs magic string.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test if btrfs.ko sucessfully identifies and reports the missing
device, if the missed device contians someother btrfs.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
I discovered a problem in btrfs where we'd end up pointing at a block we
hadn't written out yet. This is triggered by a race when two different
files on two different subvolumes fsync. This test exercises this path
with dm-log-writes, and then replays the log at every FUA to verify the
file system is still mountable and the log is replayable.
This test is to verify the fix
btrfs: fix incorrect updating of log root tree
actually fixed the problem.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add basic test to ensure btrfs conversion functionality is tested.
This test exercies conversion to all possible types of the data
portion. This is sufficient since from the POV of relocation we are
only moving blockgroups.
v5.3 and later kernel needs the following patch to pass the test
btrfs: Fix a regression which we can't convert to SINGLE profile
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add a regression test to check if btrfs can handle high devid.
The test will add and remove devices to a btrfs fs, so that the devid
will increase to uncommon but still valid values.
The regression is introduced by kernel commit ab4ba2e13346 ("btrfs:
tree-checker: Verify dev item").
The fix is titled "btrfs: tree-checker: Fix wrong check on max devid".
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Add a test case where falloc is called on multiple holes with qgroup
enabled.
This can cause qgroup reserved data space leak and false EDQUOT
error even we're not reaching the limit.
The fix is titled:
"btrfs: qgroup: Fix the wrong target io_tree when freeing
reserved data space"
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
We have generic dm-logwrites with fsstress test case (generic/482),
but it doesn't cover fs specific operations like btrfs snapshot
creation and deletion.
Furthermore, that test is not heavy enough to bump btrfs tree height
by its short runtime.
And finally, btrfs check doesn't consider dirty log as an error,
unlike ext*/xfs, that's to say we don't need to mount the fs to
replay the log, but just running btrfs check on the fs is enough.
So introduce a similar test case but for btrfs only.
The test case will stress btrfs by:
- Use small nodesize to bump tree height
- Create a base tree which is already high enough
- Trim tree blocks to find possible trim bugs
- Call snapshot creation and deletion along with fsstress
Also it includes certain workaround for btrfs:
- Allow _log_writes_mkfs to accept extra mkfs options
- Use no-holes feature
To avoid missing hole file extents.
Although that behavior doesn't follow the on-disk format spec, it
doesn't cause data loss. And will follow the new on-disk format spec
of no-holes feature, so it's better to workaround it.
And an optimization for btrfs only:
- Use replay-log --fsck/--check command
Since dm-log-writes records bios sequentially, there is no way to
locate certain entry unless we iterate all entries.
This is becoming a big performance penalty if we replay certain a
range, check the fs, then re-execute replay-log to replay another
range.
We need to records the previous entry location, or we need to
re-iterate all previous entries.
Thankfully, replay-log has already address it by providing --fsck and
--check command, thus we don't need to break replay-log command.
Please note, for fast storage (e.g. fast NVME or unsafe cache mode),
it's recommended to use log devices larger than 15G, or we can't
record the full log of just 30s fsstress run.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that an incremental send operation works after deduplicating into the
same file in both the parent and send snapshots.
This currently fails on btrfs and a kernel patch to fix it was submitted
with the subject:
Btrfs: fix incremental send failure after deduplication
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
There are two regressions related to balance resume:
- Kernel NULL pointer dereference at mount time
Introduced in v5.0
- Kernel BUG_ON() just after mount
Introduced in v5.1
The kernel fixes are:
"btrfs: qgroup: Check if @bg is NULL to avoid NULL pointer
dereference"
"btrfs: reloc: Also queue orphan reloc tree for cleanup to
avoid BUG_ON()"
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that an incremental send receive does not issue clone operations
that attempt to clone the last block of a file, with a size not aligned to
the filesystem's sector size, into the middle of some other file. Such
clone request causes the receiver to fail (with EINVAL), for kernels that
include commit ac765f83f1397646 ("Btrfs: fix data corruption due to
cloning of eof block"), or cause silent data corruption for older kernels.
This test is motived by a recent regression introduced in kernel 5.2-rc1,
commit 040ee6120cb6706 ("Btrfs: send, improve clone range"), and the
following patch fixes it:
"Btrfs: incremental send, fix emission of invalid clone perations"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that an incremental send with not corrupt data when the source
filesystem has the no-holes feature enabled, a file has prealloc
(unwritten) extents that start after its size and hole is punched (after
the first snapshot is made) that removes all extents from some offset up
to the file's size.
This currently fails on any kernel version starting from 3.16, and it's
by a patch titled:
"Btrfs: incremental send, fix file corruption when no-holes feature is enabled"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Stress send running in parallel with balance and deduplication against
files that belong to the snapshots used by send. The goal is to verify
that these operations running in parallel do not lead to send crashing
(trigger assertion failures and BUG_ONs), or send finding an inconsistent
snapshot that leads to a failure (reported in dmesg/syslog). The test
needs big trees (snapshots) with large differences between the parent and
send snapshots in order to hit such issues with a good probability.
This currently fails on btrfs, hitting a BUG_ON() often, and with btrfs
error messages in dmesg/syslog. The problem has always existed and it is
not new, but probably unnoticed due to lack of test cases that exercise
these btrfs features running in parallel.
The following patches for btrfs fix the problems:
"Btrfs: fix race between send and deduplication that lead to failures and
crashes"
"Btrfs: prevent send failures and crashes due to concurrent relocation"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Test that if we have a subvolume/snapshot that is writable, has a file
with unflushed delalloc (buffered writes not yet flushed), turn the
subvolume to readonly mode and then use it for send a operation, the send
stream will contain the delalloc data - that is, no data loss happens.
This currently files on btrfs (data loss) but is fixed by a patch for
the linux kernel titled:
"Btrfs: send, flush dellaloc in order to avoid data loss"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
We have a known bug in btrfs, that we let the device path be changed
after the device has been mounted. So using this loop hole the new
copied device would appears as if its mounted immediately after its
been copied. So this test case reproduces this issue.
For example:
Initially.. /dev/mmcblk0p4 is mounted as /
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
mmcblk0 179:0 0 29.2G 0 disk
|-mmcblk0p4 179:4 0 4G 0 part /
|-mmcblk0p2 179:2 0 500M 0 part /boot
|-mmcblk0p3 179:3 0 256M 0 part [SWAP]
`-mmcblk0p1 179:1 0 256M 0 part /boot/efi
btrfs fi show
Label: none uuid: 07892354-ddaa-4443-90ea-f76a06accaba
Total devices 1 FS bytes used 1.40GiB
devid 1 size 4.00GiB used 3.00GiB path /dev/mmcblk0p4
Copy mmcblk0 to sda
dd if=/dev/mmcblk0 of=/dev/sda
And immediately after the copy completes the change in the device
superblock is notified which the automount scans using
btrfs device scan and the new device sda becomes the mounted root
device.
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 14.9G 0 disk
|-sda4 8:4 1 4G 0 part /
|-sda2 8:2 1 500M 0 part
|-sda3 8:3 1 256M 0 part
`-sda1 8:1 1 256M 0 part
mmcblk0 179:0 0 29.2G 0 disk
|-mmcblk0p4 179:4 0 4G 0 part
|-mmcblk0p2 179:2 0 500M 0 part /boot
|-mmcblk0p3 179:3 0 256M 0 part [SWAP]
`-mmcblk0p1 179:1 0 256M 0 part /boot/efi
btrfs fi show /
Label: none uuid: 07892354-ddaa-4443-90ea-f76a06accaba
Total devices 1 FS bytes used 1.40GiB
devid 1 size 4.00GiB used 3.00GiB path /dev/sda4
The bug is quite nasty that you can't either unmount /dev/sda4 or
/dev/mmcblk0p4. And the problem does not get solved until you take
the sda out of the system on to another system to change its fsid using
the 'btrfstune -u' command.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
When a device is removed from a btrfs filesystem its superblock copies
must be deleted. This test ensures this is indeed the case.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Regression test for read corruption of compressed and shared extents
after punching holes into a file. The same extent is shared by the
same file in consecutive ranges (without other extents in between).
This is motivated by a bug recently found in btrfs for which there
is a patch for the linux kernel titled:
"Btrfs: fix corruption reading shared and compressed extents after hole
punching"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
This is a test case for a long existing bug, caused by
over-estimated metadata space_info::bytes_may_use.
There is one proposed patch for btrfs-progs to fix it, titled:
"btrfs-progs: balance: Sync the fs before balancing metadata chunks"
The test case itself is almost the same as btrfs/181, which uses
small files to bump the reserved space to trigger the false alert.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Filipe Manana
Qu Wenruo
Josef Bacik
Anand Jain
Nikolay Borisov
Christoph Hellwig