[FileCheck] Misleading prefix listed for NOT match

[hnrklssn]

❯ cat test.txt
A: hjkl
B-NOT: asdf

❯ FileCheck --check-prefixes A,B test.txt < test.txt
test.txt:2:8: error: A-NOT: excluded string found in input
B-NOT: asdf
       ^
<stdin>:2:8: note: found here
B-NOT: asdf
       ^~~~

Input file: <stdin>
Check file: test.txt

-dump-input=help explains the following input dump.

Input was:
<<<<<<
         1: A: hjkl
check:1        ^~~~
         2: B-NOT: asdf
not:2              !~~~  error: no match expected
>>>>>>

Even though the failure was caused by a matching B-NOT, the error is listed as A-NOT: excluded string found in input.

[llvmbot]

@llvm/issue-subscribers-good-first-issue

Author: Henrik G. Olsson (hnrklssn)

``` ❯ cat test.txt A: hjkl B-NOT: asdf

❯ FileCheck --check-prefixes A,B test.txt < test.txt
test.txt:2:8: error: A-NOT: excluded string found in input
B-NOT: asdf
^
<stdin>:2:8: note: found here
B-NOT: asdf
^~~~

Input file: <stdin>
Check file: test.txt

-dump-input=help explains the following input dump.

Input was:
<<<<<<
1: A: hjkl
check:1 ^~~~
2: B-NOT: asdf
not:2 !~~~ error: no match expected
>>>>>>

Even though the failure was caused by a matching `B-NOT`, the error is listed as `A-NOT: excluded string found in input`.
</details>

[Nirhar]

I would like to look into this issue, can it be assigned to me? I have some experience working with FileCheck, and would like to dive into its code with this Issue.

[hnrklssn]

Go ahead :)

[Rajveer100]

@Nirhar
Are you still working on this issue?

[Nirhar]

Hi @Rajveer100 , yes I'm working on this issue

[vinayakdsci]

Hey, @hnrklssn!
I have arrived at a solution to this issue. However, in the process, I discovered that this is the part of a bigger problem.

The erroneous code stems from llvm/lib/FileCheck/FileCheck.cpp, where in the function FileCheck::readCheckFile(), the pointer to vector CheckStrings is populated.

This is problematic in that the way it is designed, it skips all the Prefix-NOT prefixes while
populating the vector! Thus, when in FileCheck::checkInput we check for various prefixes, we are checking only for non-CHECK-DAG and non-CHECK-NOT prefixes, and hence the wrong prefix name is displayed in the error message.

Should I open a pull request for the first issue or keep digging for the bigger problem?

Thanks a lot!

EDIT: This problem is evident when the Prefix-NOT prefix is used in between the other check prefixes. When it is trailing, the bug fix is trivial.

[hnrklssn]

Small, focussed patches are encouraged here since they make it easier to review (and revert, if necessary), so if you think you found the issue, go ahead and submit a patch :) you can always submit a follow-up with more fixes later.

[vinayakdsci]

Thanks for replying!
Just one small question. How should I write the unit-test?
This is my first patch to LLVM, so I am not really sure.

Thanks a lot!

[hnrklssn]

You can use llvm/test/FileCheck/check-not-diaginfo.txt as inspiration. Let me know if you need more help!