Commit Graph

300 Commits

Author SHA1 Message Date
Eryu Guan 497363b789 fstests: introduce _exclude_scratch_mount_option helper
Some tests require that there's no certain mount option in
MKFS_OPTIONS, so introduce a new helper
_exclude_scratch_mount_option() to do the check on $MOUNT_OPTIONS.

Also convert generic/192 and xfs/134 to use this helper.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-24 12:54:01 +08:00
Eryu Guan 3832affdc9 dmthin: express low_water in cluster_size not sector size
I noticed that thin-pool sent out "reached low water mark for data
device" event even before thin-pool device was mounted in
generic/347, this is because low water mark was set to a too high
value.

According to kernel thin-provisioning.txt documentation, low water
mark should be expressed in blocks of $cluster_size, not in sectors.

"$low_water_mark is expressed in blocks of size $data_block_size."

Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-22 10:31:41 +08:00
Brian Foster 3ba258d97e fstests: support check for uuidgen utility
A recent test uses the uuidgen utility to generate UUID-based
filenames, but this package is not necessarily installed as part of
the core packages of every distro.

As such, add the UUIDGEN_PROG environment variable to the common
configuration and update the test to require the existence of
uuidgen to run.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-21 13:54:44 +08:00
Guangwen Feng 4de189ad2e shared: check corrupted orphaned inode list handling
Commit c9eb13a fixed this bug:
	ext4: fix hang when processing corrupted orphaned inode list

Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 16:35:20 +08:00
Lu Fengqi c8868d94e9 btrfs: check qgroup on extent de-reference
Test if qgroup can handle extent de-reference during reallocation.
"extent de-reference" means that reducing an extent's reference
count or freeing an extent.

Although current qgroup can handle it, we still need to prevent any
regression which may break current qgroup.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 15:55:25 +08:00
Darrick J. Wong ffad9122b6 xfs: map multiple bmbt records to a single rmapbt record
Make sure that we can handle multiple bmbt records mapping to a
single rmapbt record.  This can happen if you fallocate more than
2^21 contiguous blocks to a file.

(Also add some helpers that can create huge devices with some
dm-zero and dm-snapshot fakery.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 15:44:49 +08:00
Theodore Ts'o 850bc06d52 defrag: test $TEST_DEV for extents for ext4 defrag
Commit 902223bdbb: "defrag: require extents support for ext4
defrag" added a test to make sure the ext4 file system has extents
enabled by testing the scratch device.  Unfortunately at the time
when _require_defrag is run, the scratch file system hasn't been
initialized yet by the test, so its contents are undefined.

If the previous test explicitly creates a file system with extents
disabled on $SCRATCH_DEV (such as ext4/306), then subsequent tests
(e.g., ext4/307 and ext4/306) will refuse to run.

Fix this by testing $TEST_DEV instead of $SCRATCH_DEV.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 15:36:23 +08:00
Eric Sandeen dddc12bc76 xfs: filter stripe width details from repair
Several golden outputs have:

> Note - stripe unit (0) and width (0) fields have been reset.

but it's entirely possible for this to be non-zero,
which then fails to match and fails the test.

Filter this repair output and fix the golden files.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 15:32:18 +08:00
Zorro Lang a6f6e594f7 common/rc: teach _require_xfs_io_command accept multi-parameters
The $param can't be used for all command's options, for example
"help pwrite" include:

 -Z N -- zeed the random number generator (used when writing randomly)
         (heh, zorry, the -s/-S arguments were already in use in pwrite)

We should make param="-Z N", not only "-Z". After this patch, we can
run this function as:

  _require_xfs_io_command pwrite -Z N

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 15:29:47 +08:00
Eric Sandeen cf00fecc4f dmerror: exclude dmerror tests with mount -o dax
Same as we do for dm flakey. dm can't do dax.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Acked-by: Jeff Moyer <jmoyer@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 15:29:34 +08:00
Qu Wenruo 6df776bae8 generic: extent map search on dedupe file
For fully deduped file, which means all its file exntents are
pointing to the same bytenr, btrfs can cause soft lockup when
calling fiemap ioctl on that file, like the following output:
------
CPU: 1 PID: 7500 Comm: xfs_io Not tainted 4.5.0-rc6+ #2
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox
12/01/2006
task: ffff880027681b40 ti: ffff8800276e0000 task.ti: ffff8800276e0000
RIP: 0010:[<ffffffffa02583e4>]  [<ffffffffa02583e4>]
__merge_refs+0x34/0x120 [btrfs]
RSP: 0018:ffff8800276e3c08  EFLAGS: 00000202
RAX: ffff8800269cc330 RBX: ffff8800269cdb18 RCX: 0000000000000007
RDX: 00000000000061b0 RSI: ffff8800269cc4c8 RDI: ffff8800276e3c88
RBP: ffff8800276e3c20 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: ffff880026ea3cb0
R13: ffff8800276e3c88 R14: ffff880027132a50 R15: ffff880027430000
FS:  00007f10201df700(0000) GS:ffff88003fa00000(0000)
knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f10201ec000 CR3: 0000000027603000 CR4: 00000000000406e0
Stack:
 0000000000000000 0000000000000000 0000000000000000 ffff8800276e3ce8
 ffffffffa0259f38 0000000000000005 ffff8800274c6870 ffff8800274c7d88
 0000000000c10000 0000000000000000 0000000000000001 0000000027431190
Call Trace:
 [<ffffffffa0259f38>] find_parent_nodes+0x448/0x740 [btrfs]
 [<ffffffffa025a4f2>] btrfs_check_shared+0x102/0x1b0 [btrfs]
 [<ffffffff811fdcad>] ? __might_fault+0x4d/0xa0
 [<ffffffffa021899c>] extent_fiemap+0x2ac/0x550 [btrfs]
 [<ffffffff811ce156>] ? __filemap_fdatawait_range+0x96/0x160
 [<ffffffffa01f8ee0>] ? btrfs_get_extent+0xb30/0xb30 [btrfs]
 [<ffffffffa01f5da5>] btrfs_fiemap+0x45/0x50 [btrfs]
 [<ffffffff81246bb8>] do_vfs_ioctl+0x498/0x670
 [<ffffffff81246e09>] SyS_ioctl+0x79/0x90
 [<ffffffff8184e997>] entry_SYSCALL_64_fastpath+0x12/0x6f
Code: 41 55 41 54 53 4c 8b 27 4c 39 e7 0f 84 e9 00 00 00 49 89 fd 49 8b
34 24 49 39 f5 48 8b 1e 75 17 e9 d5 00 00 00 49 39 dd 48 8b 03 <48> 89
de 0f 84 b9 00 00 00 48 89 c3 8b 46 2c 41 39 44 24 2c 75
------

Also btrfs will return wrong flag for all these extents, they should
have SHARED(0x2000) flags, while btrfs still consider them as
exclusive extents.

On the other hand, with unmerged xfs reflink patches, xfs can handle
it without problem, and for patched btrfs, it can also handle it.

This test case will create a large fully deduped file to check if
the fs can handle the fiemap ioctl and return correct SHARED flag
for any fs which support reflink.

Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 15:09:17 +08:00
Zorro Lang 8469a8c1b3 xfs/133-4: filter redundant projid 0 quota report info out
After GETNEXTQUOTA ioctl being supported, xfs_quota -c "report"
always outputs one more quota line about default quota (as project
ID 0). In order to fix this problem, xfsprogs has merged commit
3d607a1.

Now xfstests face this same problem from this issue. xfs/133 and
xfs/134 can't match their golden output, due to this one more line
quota report output. So this patch filters this redundant quota info
out.

There're 3 kinds of xfsprogs:
1. not support GETNEXTQUOTA
2. support GETNEXTQUOTA but not merged commit 3d607a1
3. the latest version supports all

The 1st one won't report Project ID 0, the 2nd will report projid 0
info as "(null) 0 0 0 ...", the 3rd will report projid 0 info as
"#0 0 0 0 ...". To deal with all of these situations, we will use

  _filter_quota | grep -v "^#0 \|^(null) "

But if someone specifies a name for projid 0, e.g.
  # cat $projid_file
  # root:0

I think that means someone wants to deal with it by himself, the
common filter won't filter it out.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2016-06-15 15:06:57 +08:00
Omar Sandoval 0a9b318f41 btrfs: add replace missing and replace RAID 5/6 to profile configs
Replacing and scrubbing RAID 5/6 is now supported on Btrfs. Enable it in
_btrfs_get_profile_configs while making it more generic to also support
replace missing.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-05-09 10:57:50 +10:00
Xiong Zhou 8582e24f94 overlay: notrun if upper fs does not support d_type
Overlayfs whiteout can be visible if underlying upper fs does not
support d_type. Kernel commit
	45aebea (ovl: Ensure upper filesystem supports d_type)
prevents mounting overlayfs like this since v4.6-rc1.

Check upper fs before mounting overlay, mark as not run if needs.

Signed-off-by: Xiong Zhou <xzhou@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-05-09 10:55:52 +10:00
Eric Sandeen 4b52fffb5c dm-thinp helpers in common/dmthin
Basic dm-thinp helpers to set up, tear down, grow, check,
and set no-space behavior for a single thin dm volume built
on $SCRATCH_DEV.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-05-09 10:55:13 +10:00
Xiaoguang Wang f614c277ca common/populate: remove unnecessary devices directory creation
_scratch_ext4_populate and _scratch_xfs_populate try to create
devices directory in xfstests/, so if we run xfs/083 or ext4/006
twice, it'll fail, fix this issue.

Signed-off-by: Xiaoguang Wang <wangxg.fnst@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-05-09 10:52:35 +10:00
Darrick J. Wong 45da5f51d4 block: test fallocate for block devices
Now that we're wiring up fallocate's PUNCH_HOLE and ZERO_RANGE
features for block devices, add some tests to make sure they
work correctly.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-05-09 10:49:50 +10:00
Jaegeuk Kim 829b475f54 attr: adjust acl_max of f2fs if inline_xattr is set
If f2fs gets inline_xattr, it enlarges the whole xattr space, which also enables
to cover more acl entries up to 531 from 506.
This patch detects the mount option, inline_xattr, and changes the result of
_acl_get_max for generic/026.

Fixes: 9f7bf79d0c ("attr: add maximum acl count for f2fs")

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-04-05 11:48:46 +10:00
Eryu Guan 3bc4c5895c common: make sure dm device is all settled before removing it
_dmerror_cleanup()/_cleanup_flakey may fail to remove dm device due to
EBUSY and cause subsequent tests to fail.

Fix it by calling $UDEV_SETTLE_PROG before remove dm device to make sure
no one is using it.

Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-04-05 11:44:33 +10:00
Filipe Manana 2c1973ccb9 check: catch general protection fault traces in _check_dmesg
While writing a test case for overlayfs I noticed that _check_dmesg()
did not detect general protection fault traces like the following:

[15277.026108] run fstests overlay/002 at 2016-03-21 18:42:27
[15277.933179] general protection fault: 0000 [#11] PREEMPT SMP DEBUG_PAGEALLOC
[15277.935427] Modules linked in: btrfs overlay ppdev crc32c_generic evdev xor raid6_pq psmouse pcspkr sg serio_raw acpi_cpufreq parport_pc parport tpm_tis i2c_piix4
[15277.936011] CPU: 15 PID: 22762 Comm: xfs_io Tainted: G      D         4.5.0-rc6-btrfs-next-26+ #1
[15277.936011] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS by qemu-project.org 04/01/2014
[15277.936011] task: ffff88001f6f1340 ti: ffff88000a3f8000 task.ti: ffff88000a3f8000
[15277.936011] RIP: 0010:[<ffffffffa030b7ab>]  [<ffffffffa030b7ab>] btrfs_sync_file+0x11b/0x3e9 [btrfs]
[15277.936011] RSP: 0018:ffff88000a3fbe40  EFLAGS: 00010286
[15277.936011] RAX: 0000000080000000 RBX: ffff8800b0152c88 RCX: 0000000000000001
[15277.936011] RDX: 0000000000000001 RSI: ffffffff8148fec0 RDI: 00000000ffffffff
[15277.936011] RBP: ffff88000a3fbec0 R08: 0000000000000001 R09: 0000000000000000
[15277.936011] R10: ffff88000a3fbe40 R11: 0000000000000246 R12: 6e756f635f657269
[15277.936011] R13: 0000000000000000 R14: 00000000014655a0 R15: ffff88017dd25e40
[15277.936011] FS:  00007f27f0e1f700(0000) GS:ffff88023ede0000(0000) knlGS:0000000000000000
[15277.936011] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15277.936011] CR2: 00007f27f0e2f000 CR3: 000000013aadf000 CR4: 00000000000006e0
[15277.936011] Stack:
[15277.936011]  ffffffff8108b5cc ffff88000a3fbec0 0000000000000246 ffff880125abaed0
[15277.936011]  ffff8800b0152d60 8000000000000000 7fffffffffffffff 0000000000000246
[15277.936011]  0000000000000246 ffffffff81074f9b ffffffff8104357c ffff88017dd25e40
[15277.936011] Call Trace:
[15277.936011]  [<ffffffff8108b5cc>] ? arch_local_irq_save+0x9/0xc
[15277.936011]  [<ffffffff81074f9b>] ? ___might_sleep+0xce/0x217
[15277.936011]  [<ffffffff8104357c>] ? __do_page_fault+0x3c0/0x43a
[15277.936011]  [<ffffffff811a2351>] vfs_fsync_range+0x8c/0x9e
[15277.936011]  [<ffffffff811a237f>] vfs_fsync+0x1c/0x1e
[15277.936011]  [<ffffffff811a24d6>] do_fsync+0x31/0x4a
[15277.936011]  [<ffffffff811a2700>] SyS_fsync+0x10/0x14
[15277.936011]  [<ffffffff81493617>] entry_SYSCALL_64_fastpath+0x12/0x6b
[15277.936011] Code: 85 c0 0f 85 e2 02 00 00 48 8b 45 b0 31 f6 4c 29 e8 48 ff c0 48 89 45 a8 48 8d 83 d8 00 00 00 48 89 c7 48 89 45 a0 e8 fc 43 18 e1 <f0> 41 ff 84 24
[15277.936011] RIP  [<ffffffffa030b7ab>] btrfs_sync_file+0x11b/0x3e9 [btrfs]
[15277.936011]  RSP <ffff88000a3fbe40>
[15277.974598] ---[ end trace 721e14960eb939c1 ]---

So teach it to look out for such traces.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-03-23 17:39:11 +11:00
Chandan Rajendra 1ffa40385a btrfs: 'subvolume stress' tasks need to exit gracefully
The following scenario can occur when running btrfs/066,

  Task A                                Task B                     Task C

  run_test()
  - Execute _btrfs_stress_subvolume()
    in a background shell.
                                        _btrfs_stress_subvolme()
                                          ...
                                        - fork & exec "mount"
                                               	      		   Mount subvolume on directory in $TEST_DIR
  - Wait for fsstress to finish                                    do_mount()
  - kill shell process executing                                   - btrfs_mount()
    _btrfs_stress_subvolume()
    i.e. Task B.
  - Init process becomes the parent
    of "subvolume mount" task
    i.e. Task C.
  - In case subvolume is mounted
    (which is not the case),
    unmount it.
                                                                   - Complete mounting subvolume

Hence on the completion of one iteration of run_test(), the subvolume
created inside the filesystem on $SCRATCH_DEV continues to be mounted on
$TEST_DIR/$seq.mnt. Subsequent invocations of run_test() (called for
remaining Btrfs profile configs) fail during _scratch_pool_mkfs.

Instead of killing the 'subvolume stress' task this commit makes
_btrfs_stress_subvolume() to break out of the loop when a file exists
on the filesystem. The commit also makes relevant changes to other
users of _btrfs_stress_subvolume() i.e. btrfs/060, btrfs/065,
btrfs/067 & btrfs/068.

Suggested-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-03-23 17:39:11 +11:00
Eryu Guan 5af5146ef1 xfs/073: avoid finobt warning from mkfs
When testing xfs/073 with MKFS_OPTIONS="-m crc=1,finobt=1" set, it fails
due to extra warning about disabling finobt feature:

 +warning: finobt not supported without CRC support, disabled.

Because xfs/073 disables crc unconditionally and finobt can not be
enabled either.

Fix it by explicitly disabling finobt as well.

Also remove all meta related mkfs options in _scratch_mkfs_xfs_opts() if
mkfs.xfs has no metadata support, not only the crc option. So that test
still passes on distros with such old binaries.

Suggested-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-03-23 17:39:11 +11:00
Wang Xiaoguang 26ae452604 btrfs/059: add a filter for btrfs compression property
btrfs/059.out should not be hardcoded to zlib, if compression method
is lzo, this case will fail wrongly, so here add a filter.

Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-03-23 17:39:11 +11:00
David Sterba 232593dea5 filter.btrfs: new filter for property messages
The error message in newer btrfs utilities contains name of the
property, we want to filter it out so it matches golden output. There's
an optional parameter with property name that is matched against the
command output.

Signed-off-by: David Sterba <dsterba@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-03-23 14:35:14 +11:00
Ross Zwisler 1637c06780 ext4: defrag not supported with DAX
As of this kernel commit:

commit 73f34a5e2ced ("ext4: online defrag not supported with DAX")

online defrag operations for ext4 are disallowed when the filesystem is
mounted with the DAX option.

This causes several xfstests to fail because they expect the defrag
operation to change the file layout:

ext4/308	 [failed, exit status 1] - output mismatch (see /root/xfstests/results//ext4/308.out.bad)
    --- tests/ext4/308.out	2015-10-02 10:19:36.791795792 -0600
    +++ /root/xfstests/results//ext4/308.out.bad	2016-02-17 16:20:52.330454602 -0700
    @@ -23,659 +23,5 @@
     50f924a5dc9b03609a4577f9f961414b  SCRATCH_MNT/test.10
     Perform compacting
     50f924a5dc9b03609a4577f9f961414b  SCRATCH_MNT/test.10
    -Perform compacting, second pass
    -50f924a5dc9b03609a4577f9f961414b  SCRATCH_MNT/test.10
    -Create file with 20 * 2 fragments
    -wrote 1234/1234 bytes at offset 0
    ...
    (Run 'diff -u tests/ext4/308.out /root/xfstests/results//ext4/308.out.bad'  to see the entire diff)
generic/018 1s ... [failed, exit status 1] - output mismatch (see /root/xfstests/results//generic/018.out.bad)
    --- tests/generic/018.out	2016-02-17 16:02:40.103656140 -0700
    +++ /root/xfstests/results//generic/018.out.bad	2016-02-17 16:20:53.117459173 -0700
    @@ -10,10 +10,6 @@
     After: 1
     Write backwards sync, but contiguous - should defrag to 1 extent
     Before: in_range(5, 10)
    -After: 1
    -Write backwards sync leaving holes - defrag should do nothing
    -Before: 16
    -After: 16
    ...
    (Run 'diff -u tests/generic/018.out /root/xfstests/results//generic/018.out.bad'  to see the entire diff)

Avoid this by skipping over defrag tests if we are using ext4 + DAX.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
2016-03-23 14:33:40 +11:00