Substitutions are already reported in the diagnostics appearing before
the input dump in the case of failed directives, and they're reported
in traces (produced by `-vv -dump-input=never`) in the case of
successful directives. However, those reports are not always
convenient to view while investigating the input dump, so this patch
adds the substitution report to the input dump too. For example:
```
$ cat check
CHECK: hello [[WHAT:[a-z]+]]
CHECK: [[VERB]] [[WHAT]]
$ FileCheck -vv -DVERB=goodbye check < input |& tail -8
<<<<<<
1: hello world
check:1 ^~~~~~~~~~~
2: goodbye word
check:2'0 X~~~~~~~~~~~ error: no match found
check:2'1 with "VERB" equal to "goodbye"
check:2'2 with "WHAT" equal to "world"
>>>>>>
```
Without this patch, the location reported for a substitution for a
directive match is the directive's full match range. This location is
misleading as it implies the substitution itself matches that range.
This patch changes the reported location to just the match range start
to suggest the substitution is known at the start of the match. (As
in the above example, input dumps don't mark any range for
substitutions. The location info in that case simply identifies the
right line for the annotation.)
Reviewed By: mehdi_amini, thopre
Differential Revision: https://reviews.llvm.org/D83650
In FileCheck.rst, add `-dump-input-context` and `-dump-input-filter`,
and fix some `-dump-input` documentation.
In `FileCheck -help`, `cl::value_desc("kind")` is being ignored for
`-dump-input-filter`, so just drop it.
Extend `-dump-input=help` to mention FILECHECK_OPTS.
This makes the input dump filtering implemented by D82203 more
configurable. D82203 enables filtering out everything but the initial
input lines of error diagnostics (plus some context). This patch
enables including any line with any kind of annotation.
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D83097
For example, given `-dump-input-context=3 -vv`, the following now
shows more leading context for the error than requested because a
leading ellipsis would occupy the same number of lines as it would
elide:
```
<<<<<<
1: foo6
2: foo5
3: foo4
4: foo3
5: foo2
6: foo1
7: hello world
check:1 ^~~~~
check:2 X~~~~ error: no match found
8: foo1
check:2 ~~~~
9: foo2
check:2 ~~~~
10: foo3
check:2 ~~~~
.
.
.
>>>>>>
```
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D83526
This patch is motivated by discussions at each of:
* <https://reviews.llvm.org/D81422>
* <http://lists.llvm.org/pipermail/llvm-dev/2020-June/142369.html>
When input is dumped as specified by `-dump-input=fail`, this patch
filters the dump to show only input lines that are the starting lines
of error diagnostics plus the number of contextual lines specified
`-dump-input-context` (defaults to 5).
When `-dump-input=always`, there might be not be any errors, so all
input lines are printed, as without this patch.
Here's some sample output with `-dump-input-context=3 -vv`:
```
<<<<<<
.
.
.
13: foo
14: foo
15: hello world
check:1 ^~~~~~~~~~~
16: foo
check:2'0 X~~ error: no match found
17: foo
check:2'0 ~~~
18: foo
check:2'0 ~~~
19: foo
check:2'0 ~~~
.
.
.
27: foo
check:2'0 ~~~
28: foo
check:2'0 ~~~
29: foo
check:2'0 ~~~
30: goodbye word
check:2'0 ~~~~~~~~~~~~
check:2'1 ? possible intended match
31: foo
check:2'0 ~~~
32: foo
check:2'0 ~~~
33: foo
check:2'0 ~~~
.
.
.
>>>>>>
```
Reviewed By: mehdi_amini, arsenm, jhenderson, rsmith, SjoerdMeijer, Meinersbur, lattner
Differential Revision: https://reviews.llvm.org/D82203
Document the default of `fail` in `-help`. Extend `-dump-input=help`
to help users find related command-line options, but let `-help`
provide their full documentation.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D83091
`FILECHECK_OPTS` was implemented so that a test runner, such as CI,
can specify FileCheck debugging options, such as `-v` and `-vv`.
However, if a test suite has a FileCheck call that already specifies
`-v` or `-vv`, then that call will fail if `FILECHECK_OPTS` also
specifies it.
For `-vv`, this problem already exists:
`clang/test/CodeGen/aarch64-v8.2a-fp16-intrinsics-constrained.c`
It's not yet clear if the `-vv` in that test was intentional, but this
usage shouldn't fail anyway. It's already true that FileCheck permits
`-vv` and `-v` together even though `-vv` implies `-v`.
Compare D70784, which fixed the same problem for `-dump-input`.
Reviewed By: jhenderson, thopre
Differential Revision: https://reviews.llvm.org/D82601
Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which really helps to understand failures.
Remove `-dump-input-on-failure` and the environment variable
FILECHECK_DUMP_INPUT_ON_FAILURE which are now obsolete.
Differential Revision: https://reviews.llvm.org/D81422
Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` directive makes it easy to do this. For example,
you might have:
```
; X32: pinsrd_1:
; X32: pinsrd $1, 4(%esp), %xmm0
; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM: X64: pinsrd_1:
; COM: X64: pinsrd $1, %edi, %xmm0
```
Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:
<http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>
I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear. `COM:` can avoid all these problems.
This patch also updates the small set of existing tests that define
`COM` as a check prefix:
- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll
I think lit should support `COM:` as well. Perhaps `clang -verify`
should too.
Reviewed By: jhenderson, thopre
Differential Revision: https://reviews.llvm.org/D79276
This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.
Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.
Reviewed By: jhenderson, thopre
Differential Revision: https://reviews.llvm.org/D79375
Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` directive makes it easy to do this. For example,
you might have:
```
; X32: pinsrd_1:
; X32: pinsrd $1, 4(%esp), %xmm0
; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM: X64: pinsrd_1:
; COM: X64: pinsrd $1, %edi, %xmm0
```
Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:
<http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>
I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear. `COM:` can avoid all these problems.
This patch also updates the small set of existing tests that define
`COM` as a check prefix:
- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll
I think lit should support `COM:` as well. Perhaps `clang -verify`
should too.
Reviewed By: jhenderson, thopre
Differential Revision: https://reviews.llvm.org/D79276
This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.
Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.
Reviewed By: jhenderson, thopre
Differential Revision: https://reviews.llvm.org/D79375
There are few `std::vector<std::string>` members in
`FileCheckRequest`. This patch changes these arrays to `std::vector<StringRef>`
and refactors the code related to cleanup/improve/simplify it.
Differential revision: https://reviews.llvm.org/D78202
Without this patch, `--dump-input` annotations on a single input line
are sorted by the associated directive's check-file line. That seemed
fine because that's often identical to the order in which FileCheck
looks for matches for those directives.
The first problem is that an `--implicit-check-not` pattern has no
check-file line. The logical equivalent is sorting in command-line
order, but that's not implemented.
The second problem is that, unlike a directive, an
`--implicit-check-not` pattern applies at many points, between many
different pairs of directives. However, sorting in command-line order
gathers all its associated diagnostics together at one point in an
input line's list of annotations.
In general, it seems to be easier to understand FileCheck's logic when
annotations on a single input line are sorted in the order FileCheck
produced the associated diagnostics, so this patch makes that change.
As documented in the patch, the annotation sort order is also
especially relevant to `CHECK-LABEL`, `CHECK-NOT`, and `CHECK-DAG`, so
this patch updates or extends tests to check the sort makes sense for
them. (However, the sort for `CHECK-DAG` annotations should not
actually be altered by this patch.)
Reviewed By: thopre
Differential Revision: https://reviews.llvm.org/D77607
Currently, `--dump-input` implies that all `--implicit-check-not`
patterns appear on line 1 by printing annotations like:
```
1: foo bar baz
not:1 !~~ error: no match expected
```
This patch changes that to:
```
1: foo bar baz
not:imp1 !~~ error: no match expected
```
`imp1` indicates the first `--implicit-check-not` pattern.
Reviewed By: thopre
Differential Revision: https://reviews.llvm.org/D77605
Summary: Replace '-' in the error message with <stdin>. This is also consistent with another error message in the code.
Reviewers: jhenderson, probinson, jdenny, grimar, arichardson
Reviewed By: jhenderson
Subscribers: thopre, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73793
Problem: `FILECHECK_OPTS` was implemented so that a test runner, such
as a bot, can specify FileCheck debugging options, such as
`-dump-input=fail`. However, some existing test suites have FileCheck
calls that already specify `-dump-input=fail` or `-dump-input=always`.
Without this patch, such tests fail under such a test runner because
FileCheck doesn't accept multiple occurrences of `-dump-input`.
Solution: This patch permits multiple occurrences of `-dump-input` by
assigning precedence to its values in the following descending order:
`help`, `always`, `fail`, and `never`. That is, any occurrence of
`help` always obtains help, and otherwise the behavior is similar to
`-v` vs. `-vv` in that the option specifying the greatest verbosity
has precedence.
Rationale: My justification for the new behavior is as follows. I
have not experienced use cases where, either as a test runner or as a
test author, I want to **limit** the permitted debugging verbosity
(except as a test author in FileCheck's or lit's test suites where the
FileCheck debugging output itself is under test, but the solution
there is `env FILECHECK_OPTS=`, and I imagine we should use the same
solution anywhere else this need might occur). Of course, as either a
test runner or test author, it is useful to **increase** debugging
verbosity.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D70784
The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).
Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D68146
llvm-svn: 374538
The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).
Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D68146
llvm-svn: 374339
Summary:
Remove use of FileCheckPatternContext and FileCheckString concrete types
from FileCheck API to allow moving it and the other implementation only
only declarations into a private header file.
Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk
Subscribers: hiraditya, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D68186
llvm-svn: 373211
Joel E. Denny
Mehdi Amini
Georgii Rymar
David Bozier
Kai Nacke
Dmitri Gribenko
Thomas Preud'homme