apfs/apfstests
0
0
Fork 0
mirror of https://github.com/linux-apfs/apfstests.git synced 2026-05-01 15:01:44 -07:00
Code Issues Packages Projects Releases Wiki Activity

generic/038: speed up file creation

Browse Source 
Now that generic/038 is running on my test machine, I notice how
slow it is:

generic/038      692s

11-12 minutes for a single test is way too long.
The test is creating
400,000 single block files, which can be easily parallelised and
hence run much faster than the test is currently doing.

Split the file creation up into 4 threads that create 100,000 files
each. 4 is chosen because XFS defaults to 4AGs, ext4 still has decent
speedups at 4 concurrent creates, and other filesystems aren't hurt
by excessive concurrency. The result:

generic/038      237s

on the same machine, which is roughly 3x faster and so it (just)
fast enough to to be considered acceptible.

[Eryu Guan: reduced number of files to minimum needed to reproduce
 btrfs problem reliably, added $LOAD_FACTOR scaling for longer
 running.]

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Dave Chinner <david@fromorbit.com>
This commit is contained in:
Dave Chinner
2015-09-21 11:03:22 +10:00
committed by Dave Chinner
parent e1efba7d17
commit c9b9d5f6a9
1 changed files with 19 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/generic/038
+19 -7
View File
@@ -105,19 +105,31 @@ trim_loop()
# the fallocate calls happen. So we don't really care if they all succeed or
# not, the goal is just to keep metadata space usage growing while data block
# groups are deleted.
#
# Creating 200,000 files sequentially is really slow, so speed it up a bit
# by doing it concurrently with 4 threads in 4 separate directories.
nr_files=$((50000 * LOAD_FACTOR))
create_files()
{
local prefix=$1
for ((i = 1; i <= 400000; i++)); do
$XFS_IO_PROG -f -c "pwrite -S 0xaa 0 3900" \
$SCRATCH_MNT/"${prefix}_$i" &> /dev/null
if [ $? -ne 0 ]; then
echo "Failed creating file ${prefix}_$i" >>$seqres.full
break
fi
for ((n = 0; n < 4; n++)); do
mkdir $SCRATCH_MNT/$n
(
for ((i = 1; i <= $nr_files; i++)); do
$XFS_IO_PROG -f -c "pwrite -S 0xaa 0 3900" \
$SCRATCH_MNT/$n/"${prefix}_$i" &> /dev/null
if [ $? -ne 0 ]; then
echo "Failed creating file $n/${prefix}_$i" >>$seqres.full
break
fi
done
) &
create_pids[$n]=$!
done
wait ${create_pids[@]}
}
_scratch_mkfs >>$seqres.full 2>&1