rust-analyzer reader-mode

[matklad]

TL;DR: https://github.com/rust-dev-tools/cargo-src, but on top of rust-analyzer

Add a command, rust-analyzer reader-mode path/to/Cargo.toml, which produces a directory with a bunch of html files, which contain source for the project, a-la rustdoc output, but also have goto definition, hover, etc.

Interface Constraints:

• Static site
• Assume modern desktop browser
• No external CSS, or CSS pre-processors, minimal hand-written CSS to get the job done.
• No or minimal JavaScript
• No build-in HTTP serever (but it's OK to try to auto-discover python3 -m http.server and such on usres' machine)
• (Maybe) an option to compress everything down to a single html file which you can just open in the browser.

Implementation Constraints:

• Implementatin lives in ide-reader-mode and is exposed via ide crate.
• There's a transparent intermediate representation. The result of reader mode is not directly an HTML string, but something else. That something else is a set of annotated files, struct ReaderView. This view is independent of the hir or ide types, and is just a large chunk of data. The data includes two things, a list of files and a list of defs. Each file is a text plus a number of auxilary maps that describe ranges in the file. There should be a map for syntax hightlighting, a map for references and definitons, maybe a map for folding ranges (fold method bodies would be useful feature to have). Spans in files refer to defs by ID. Defs are untyped, and store their defining spans and a list of refs. Not sure if we need a separate ref list or just spans would be enough.
• ReaderView might want to live in a separate crate, just to make sure it doesn't depend on rust-analyzer types
• ReaderView has fn to_html(&self) -> HashMap<PathBuf, String> method, which renders it as a directory with htmls and csses. This should be completely independent of the rust-analyzer.

[edwin0cheng]

There's a transparent intermediate representation ..

Should it store in LSIF ?

[matklad]

I think it’s better not to, it seems simpler to have a thing taylored specifically for rust and code browsing

[DJMcNab]

We probably should also have LSIF support, because my guess is the GitHub's code analysis engine which they use for providing support for go to definition in other languages uses.

[matklad]

I do think we should have LSIF, but I think that's orthogonal feature .

[DJMcNab]

Yeah, I think we're in agreement there. In theory, the LSIF dump would probably use a lot of similar infrastructure.

[jonas-schievink]

GitHub's code navigation uses https://github.com/github/semantic

[flodiebold]

Seems a bit of a weird choice for them to try to implement language analysis for a bunch of languages themselves 😕

[DJMcNab]

That would at least explain why it can be so inconsistent though, I guess.

[matklad]

This drifting oftopic, but I actually see reasons for them to do their own thing. They have a boatload of code, and they can't build it (both in terms of resources, and in terms of each project requiring a specific environment to build). In this scenario, building a distributed version on IntelliJ architecture makes sense. It would not be precise, but it would be robust.

If you have a boatload of code, but you can build it (because, for example, you are Google, so there's unified Bazel build system), then language-specific Kythe approach makes sense.

[flodiebold]

Spans in files refer to defs by ID. Defs are untyped, and store their defining spans and a list of refs.

Why have def IDs at all, and not just refer to everything by span?

[flodiebold]

a list of files

I imagine this might include virtual "macro expansion" files so we can navigate into those 🤔