Files
apfstests/doc/requirement-checking.txt
T
Josef Bacik c134a345f7 fstests: add fio perf results support
This patch does the nuts and bolts of grabbing fio results and
storing them in a database in order to check against for future
runs.  This works by storing the results in resuts/fio-results.db as
a sqlite database.  The src/perf directory has all the supporting
python code for parsing the fio json results, storing it in the
database, and loading previous results from the database to compare
with the current results.

This also adds a PERF_CONFIGNAME option that must be set for this to
work.  Since we all have various ways we run fstests it doesn't make
sense to compare different configurations with each other (unless
specifically desired).  The PERF_CONFIGNAME will allow us to
separate out results for different test run configurations to make
sure we're comparing results correctly.

Currently we only check against the last perf result.  In the future
I will flesh this out to compare against the average of N number of
runs to be a little more complete, and hopefully that will allow us
to also watch latencies as well.

[eguan: add required Makefile updates]

Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
2017-11-26 15:37:22 +08:00

134 lines
3.5 KiB
Plaintext

========================================
TESTING FOR REQUIREMENTS IN TEST SCRIPTS
========================================
Test scripts need to indicate to the infrastructure what sorts of requirements
they have. This is done with _require_<xxx> macros, which may take parameters.
(1) General requirements.
_require_command "$<NAME_PROG>" <name>
_require_test
_require_test_program <name>
_require_xfs_io_command <name> [<switch>]
(2) Filesystem capability requirements.
_require_chattr <letters>
_require_exportfs
(3) System call requirements.
_require_statx
(4) Device mapper requirement.
_require_dm_target
_require_log_writes
====================
GENERAL REQUIREMENTS
====================
_require_command "$NAME_PROG" name
The test requires an external command, called 'name' be present on the
system and that '$VAR' should be set with the path to that command. $VAR
should then be used to refer to the command when executing it. For
example:
_require_command "KILLALL_PROG" killall
to locate the killall command and then:
$KILLALL_PROG -q $FSSTRESS_PROG
to make use of it.
_require_test
The test requires that the block device specified by $TEST_DEV be mounted
on $TEST_DIR.
_require_test_program <name>
The test requires a program by the name of 'name' be present and built in
the src/ directory. For example:
_require_test_program "stat_test"
requires that src/stat_test be built.
_require_xfs_io_command <name> [<switch>]
The test requires that the xfs_io command be available, that it supports
command <name> and, optionally, that that command supports the specified
switch. For example:
_require_xfs_io_command "falloc"
_require_xfs_io_command "chattr" "+/-x"
The first requires that xfs_io support the falloc command and the second
that it supports the chattr command and that the chattr command supports
the +x and -x arguments (DAX attribute).
==================================
FILESYSTEM CAPABILITY REQUIREMENTS
==================================
_require_chattr <letters>
The test requires that the filesystem attribute set by the chattr command
with +<letters> as an argument be available and supported by the $TEST_DEV
filesystem. No check is made of the scratch filesystem. For example:
_require_chattr ai
tests to see if setting the append-only and immutable attributes on a file
(chattr +a +i) is supported.
_require_exportfs
The test requires that the $TEST_DEV filesystem supports NFS export.
The test also requires the use of the open_by_handle_at() system call and
will be skipped if it isn't available in the kernel.
========================
SYSTEM CALL REQUIREMENTS
========================
_require_statx
The test requires the use of the statx() system call and will be skipped
if it isn't available in the kernel.
==========================
DEVICE MAPPER REQUIREMENTS
==========================
_require_dm_target <name>
The test requires the use of the device mapper target and will be skipped
if it isn't available in the kernel.
_require_log_writes
The test requires the use of the device mapper target log-writes.
The test also requires the test program log-writes/replay-log is built
and will be skipped if either isn't available.
======================
PERF TEST REQUIREMENTS
======================
_require_fio_results
This test requires the supporting tools for saving and comparing fio based
perf test results.