mirror of
https://github.com/linux-apfs/apfstests.git
synced 2026-05-01 15:01:44 -07:00
Eric Sandeen
2ad5167475
This is a simple script to compare failures across runs.
Given files containing stdout from several runs, each of which contains
a Failures: line, it will print a table of all failures for each run.
Test subdir names are abbreviated for compactness, i.e. generic->g.
For 7 results files named test 1 through test 7:
Failures:
g/075 g/082 g/209 g/233 g/270 g/388 x/004 x/073 x/076
-----------------------------------------------------
g/082 g/233 x/004 x/073 test1
g/082 g/233 x/004 x/073 x/076 test2
g/082 x/004 x/073 x/076 test3
g/082 g/388 x/004 x/073 test4
g/082 g/270 x/004 x/073 test5
g/082 x/004 x/073 test6
g/075 g/082 g/209 g/233 x/004 x/073 test7
This lets us easily spot unique failures and outliers.
This could be enhanced to output CSV etc, but for now I think it's
helpful to visualize changes in failures across multiple runs.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
68 lines
1.9 KiB
Bash
Executable File
68 lines
1.9 KiB
Bash
Executable File
#!/bin/bash
|
|
|
|
# Compare test failures across runs
|
|
#
|
|
# Takes multiple "results" files as arguments, comprised of the
|
|
# stdout from a ./check run, each containing a Failures: line.
|
|
#
|
|
# Outputs a table of failures for comparison across runs
|
|
#
|
|
#-----------------------------------------------------------------------
|
|
# Copyright (c) 2017 Red Hat, Inc. All Rights Reserved.
|
|
#
|
|
# This program is free software; you can redistribute it and/or
|
|
# modify it under the terms of the GNU General Public License as
|
|
# published by the Free Software Foundation.
|
|
#
|
|
# This program is distributed in the hope that it would be useful,
|
|
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
# GNU General Public License for more details.
|
|
#
|
|
# You should have received a copy of the GNU General Public License
|
|
# along with this program; if not, write the Free Software Foundation,
|
|
# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
|
#-----------------------------------------------------------------------
|
|
#
|
|
|
|
filter_names() {
|
|
sed -e s/btrfs/b/ \
|
|
-e s/cifs/c/g \
|
|
-e s/f2fs/f/g \
|
|
-e s/generic/g/g \
|
|
-e s/overlay/o/g \
|
|
-e s/shared/s/g \
|
|
-e s/udf/u/g \
|
|
-e s/xfs/x/g
|
|
}
|
|
|
|
# ALLFAILURES: A B C D E F G
|
|
# THESEFAILURES: A C D G
|
|
|
|
# We want to print the header (ALLFAILURES) and then
|
|
# if a run didn't fail a particular test, print spaces instead
|
|
|
|
# All tests that failed in any run, all on one line, unique
|
|
ALLFAILURES=`grep -h ^Failures: $* \
|
|
| tr " " "\n" \
|
|
| sort | uniq \
|
|
| filter_names \
|
|
| tr "\n" " " \
|
|
| sed -e "s/^Failures: //g"`
|
|
|
|
# Header
|
|
echo "Failures:"
|
|
echo $ALLFAILURES
|
|
echo $ALLFAILURES | sed -e "s/./-/g"
|
|
|
|
# Per-file failures
|
|
for FILE in $*; do
|
|
THESEFAILURES=`grep ^Failures: $FILE | filter_names`
|
|
for FAILURE in $ALLFAILURES; do
|
|
CELL=`echo $THESEFAILURES \
|
|
| grep -wo "$FAILURE" || echo $FAILURE | sed -e "s/./ /g"`
|
|
echo -n "$CELL "
|
|
done
|
|
echo $FILE
|
|
done
|