mirror of
https://github.com/linux-apfs/apfstests.git
synced 2026-05-01 15:01:44 -07:00
Eric Sandeen
2ad5167475
This is a simple script to compare failures across runs.
Given files containing stdout from several runs, each of which contains
a Failures: line, it will print a table of all failures for each run.
Test subdir names are abbreviated for compactness, i.e. generic->g.
For 7 results files named test 1 through test 7:
Failures:
g/075 g/082 g/209 g/233 g/270 g/388 x/004 x/073 x/076
-----------------------------------------------------
g/082 g/233 x/004 x/073 test1
g/082 g/233 x/004 x/073 x/076 test2
g/082 x/004 x/073 x/076 test3
g/082 g/388 x/004 x/073 test4
g/082 g/270 x/004 x/073 test5
g/082 x/004 x/073 test6
g/075 g/082 g/209 g/233 x/004 x/073 test7
This lets us easily spot unique failures and outliers.
This could be enhanced to output CSV etc, but for now I think it's
helpful to visualize changes in failures across multiple runs.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
Quick guide to auto-qa dxm 04/10/2000
______________________ ______________
- pick/create a user to run auto-qa and check they
can use ptools to check out of the tree
- add your host to xfstests/common/config and check this file in
OR create a local.config file in your xfstests directory
- make a directory "$HOME/qa"
- make a kernel workarea "$HOME/qa/2.4.x-xfs" for 2.4.x-xfs
and a commands workarea "$HOME/qa/xfs-cmds" for xfs-cmds
(easiest to copy them from elsewhere)
- copy an appropriate .config file to
$HOME/qa/$HOSTNAME.config
- You'll need a hacked version of 'su' in $HOME/qa that
lets your user su to root/root without a password
(if you want to run from cron, it mustn't require
/dev/tty). Warning - this is a massive security
hole.
- chown root.root $HOME/qa/su
- chmod 6755 $HOME/qa/su
- add the soon to be kernel to /etc/lilo.conf
image=/boot/vmlinuz-xfs-qa
label=linux-xfs-qa
append = "console=ttyS0,38400n8"
- $HOME/qa/auto-qa init
At this point, the script should update the workarea, clean it,
rebuild it, install it and reboot.
Then run
- $HOME/qa/auto-qa restarted
And the tests should happen... and all pass, of course.
To get it going automagically:
- add some lines to the appropriate user's crontab:
0 4 * * * $HOME/qa/auto-qa cron-init
30 4 * * * $HOME/qa/auto-qa cron-restarted
Notes:
- if MODULAR=1 in auto-qa the XFS filesystem is expected to
be a module; if MODULAR=0 it should be built into
the kernel.
- the test device is cleaned at the start of the QA run
(to stop nightly QA being stuffed up if someone
leaves the device inconsistant etc)
- I'm using a hacked su because PCP sudo won't set the
gid properly, and normal linux su won't run
without a tty even if PAM is switched off.
- The QA is restarted after reboot by a second cron entry
to avoid the test being able to get itself into
some stupid loop and so that it's always started
by the appropriate user. You might have to make
the second cron run later if your build takes ages.
- Point the email addresses somewhere appropriate
- When run in "cron-init" or "init" states, the script
will p_tupdate itself and restart. If you start
with an empty source tree, you'll need to check
out the xfstests/tools/auto-qa script before it'll
work (duh).
good luck.