fix: handle escaped chars in doc comments

[roife]

fix #16980.

For ast::LiteralKind::String, store the original string value.

[roife]

cc @Veykril See #16980 (comment)

I'm not sure if this is the best approach to handle escaped chars, but if we store text instead of not storing the string value, then we'll need to unescape it later on, which makes it complex. Hope you can provide some suggestions.

[Veykril]

This won't work, both our mbe parsing and the proc-macro logic expect tt::Literals to actually contain literal values, which in the case of strings would be a proper string literal value. I'd prefer if we could store the text verbatim so the span fits (ideally this will make syntax injection work properly for doc attributes, I haven't gotten to that yet though).

I don't think having to unescape at call sites should be too bad though, we can have the string_value function do just that (or have a separate one if they are callees that don't care about it).

[roife]

If we unescape characters in string_value, then we have to return an Option<String> or Option<Box<str>> (instead of the original Option<&str>), which will affect many functions in the calling chain.

[Veykril]

That's fine, we can return a Cow<str> (and replace invalid escapes with something) and have two functions string_value -> &str and string_value_escaped -> Cow<str>, alomst all callers don't care about escapes. The important ones seem to be the user facing ones.

[Veykril]

That is, in https://github.com/rust-lang/rust-analyzer/blob/4135696ea7d5b09f30f79676a3795f91610a025d/crates/ide-db/src/documentation.rs there are some calls that care about unescaping and I presume (haven't checked),

rust-analyzer/crates/hir-def/src/nameres/collector.rs

Line 1920 in 4135696

let path_attr = attrs.by_key("path").string_value();

this one also cares about unescaping. The rest does not seem to care so we can keep the escaped versions there.

[Veykril]

Let's re-export the unescape stuff from the parser or syntax crate (reason being that the ra-ap-rustc crates are special and require extra extern crate prelude annotations, a re-export saves us from that here

[Veykril]

One more annoyance, but otherwise lgtm! Thanks!
@bors delegate+

[bors]

✌️ @roife, you can now approve this pull request!

If @Veykril told you to "r=me" after making some further change, please make that change, then do @bors r=@Veykril

[Veykril]

Thanks!
@bors r+

[bors]

📌 Commit 3e232bb has been approved by Veykril

It is now in the queue for this repository.

[bors]

⌛ Testing commit 3e232bb with merge 05428c5...

[bors]

☀️ Test successful - checks-actions
Approved by: Veykril
Pushing 05428c5 to master...