Add parse_to_token_tree

[edwin0cheng]

This PR introduce a function for parsing &str to tt::TokenTree:

// Convert a string to a `TokenTree`
pub fn parse_to_token_tree(text: &str) -> Option<(tt::Subtree, TokenMap)> {

[matklad]

Hm, I am somewhat unhappy about the duplication of logic here.... In rustc, all parsing happens through the syntax trees, so you get a nice syntax pipeline.

We parse either from token trees or from text, and those two modes are different, and require explicit bridging between them. And include! case becomes kind of silly, because it seems like you should use "parse text" mode, but, because it is a macro, you have to interface with it via token trees.... Not sure if there's a good solution here....

bors r+

[bors]

Build succeeded

• Rust (ubuntu-latest)
• Rust (windows-latest)
• Rust (macos-latest)
• TypeScript