Skip to content

unexpected token in input when proptest! in file #6747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Eh2406 opened this issue Dec 7, 2020 · 10 comments · Fixed by #6874
Closed

unexpected token in input when proptest! in file #6747

Eh2406 opened this issue Dec 7, 2020 · 10 comments · Fixed by #6874
Assignees
Labels
A-macro macro expansion S-actionable Someone could pick this issue up and work on it right now

Comments

@Eh2406
Copy link

Eh2406 commented Dec 7, 2020

I just updated my rust analyzer, and now every file (at least in this project) gets the error unexpected token in input. The project is https://github.com/pubgrub-rs/pubgrub, and @lnicola can reproduce. They think it is the use of proptest!. I don't know how to be more helpful.

cc https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/unexpected.20token.20in.20input

@jonas-schievink jonas-schievink added the A-macro macro expansion label Dec 8, 2020
@jonas-schievink jonas-schievink self-assigned this Dec 8, 2020
@crumblingstatue
Copy link

crumblingstatue commented Dec 8, 2020

I don't know if it's the same bug, but my advent of code project also has unexpected token in input in every solution in src/bin because of the macro usage. And they all have incorrect span information, pointing to the first line.
https://github.com/crumblingstatue/advent-of-code

@Eh2406
Copy link
Author

Eh2406 commented Dec 8, 2020

BTW the suggestion here https://www.reddit.com/r/rust/comments/k8fo7w/rustanalyzer_changelog_54/gexzaem/?utm_source=reddit&utm_medium=web2x&context=3 got the meseges to go way so I can use rust-analyzer untill this is fixed.

@tensor-programming
Copy link

tensor-programming commented Dec 8, 2020

Had the same issue; can confirm that the suggestion removes the problem for now. In my case, the error was could not convert tokens and this is for a very simple macro that just uses concat!, file!() and line!() to create a simple error code.

#[macro_export]
macro_rules! line_error {
    () => {
        concat!("Error at ", file!(), ":", line!())
    };
    ($str:expr) => {
        concat!($str, " @", file!(), ":", line!())
    };
}

At least doing this gets rid of all of the red in my project:

"rust-analyzer.diagnostics.disabled": [
    "macro-error"
  ]

@jonas-schievink
Copy link
Contributor

Heh, looks like that's because concat! doesn't like the integer produced by line!().

bors bot added a commit that referenced this issue Dec 8, 2020
6765: Fix file range computation in macros r=jonas-schievink a=jonas-schievink

This also aligns the diagnostics behavior of `TestDB` with the one from the real IDE (by making the logic from `semantics.rs` a method on `InFile<&SyntaxNode>`), which makes bugs like this easier to find.

This should fix the misplaced diagnostics seen in #6747 and other issues.

bors r+

Co-authored-by: Jonas Schievink <[email protected]>
bors bot added a commit that referenced this issue Dec 8, 2020
6768: Fix `concat!` with integer literals r=jonas-schievink a=jonas-schievink

Addresses #6747 (comment)

bors r+

Co-authored-by: Jonas Schievink <[email protected]>
@jonas-schievink
Copy link
Contributor

The concat! issue, as well as the misplaced diagnostics, are now fixed.

@lnicola
Copy link
Member

lnicola commented Dec 9, 2020

This seems to work fine now:

image

There might still be some resolution errors on the enum variants in Term::is_positive, but that's a different issue.

@lnicola lnicola closed this as completed Dec 9, 2020
@Eh2406
Copy link
Author

Eh2406 commented Dec 14, 2020

I just updated my version of rust-analyzer and removed the change in settings. And this is still happening. It is no longer every file, it is no longer at the top. But every call to proptest! gets unexpected token in input.

@lnicola lnicola reopened this Dec 14, 2020
@lnicola
Copy link
Member

lnicola commented Dec 14, 2020

Repro:

macro_rules! foo {
    () => {{
        concat!(module_path!(), "::");
    }};
}

fn main() {
    foo! {}
}

@lnicola lnicola added the S-actionable Someone could pick this issue up and work on it right now label Dec 14, 2020
@jonas-schievink
Copy link
Contributor

Even just concat!(module_path!(), "::"); is enough

@jonas-schievink
Copy link
Contributor

Oh, we simply do not implement module_path!().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-macro macro expansion S-actionable Someone could pick this issue up and work on it right now
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants