Introduce ra_proc_macro

[edwin0cheng]

This PR implemented:

1. Reading dylib path of proc-macro crate from cargo check , similar to how OUTDIR is obtained.
2. Added a new crate ra_proc_macro and implement the foot-work for reading result from external proc-macro expander.
3. Added a struct ProcMacroClient , which will be responsible to the client side communication to the External process.

[edwin0cheng]

cc @kiljacken for the dylib path reading part :)

[matklad]

I'd like to pay some attention to minimizing the crate deps here.

Ideally, the ra_db should not depend only any proc-macro specfic, and just export a trait. It should operate on tt::TokenStream, so a dep on ra_tt is required.

Additionally, Ideally proc_macros and mbe do not depend on each other, and use solely tt (which does not depen on ra_syntax). I think we should also make use that proc_macros dont (transitivellly) depend on ra_syntax.

Not sure if the above plan is feasible, but i'd love to try

[matklad]

Hm, I wonder if we can minimize the interface here...

Can we use something like this instead?

struct CrateData {
   proc_macros: FxHashMap<SmolStr, Arc<dyn Fn(tt::TokenTree) -> tt::TokenTree>>
}

Ie, define a really minimal proc macro interface directly in the ra_db crate, where the minimal interface is ideally an dyn Fn(), or maybe a named trait.

[edwin0cheng]

I changed to :

#[derive(Debug, Clone)]
pub struct ProcMacro {
    pub name: SmolStr,
    pub expander: Arc<dyn TokenExpander>,
}

impl Eq for ProcMacro {}
impl PartialEq for ProcMacro { ... }

#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CrateData {
   ...
    pub proc_macro: Vec<ProcMacro>,
}

[matklad]

Also, thanks @edwin0cheng for splitting changes which are small diff-wise, but are large architecturally, into separate PRs! This is massively helpful!

[edwin0cheng]

Ideally, the ra_db should not depend only any proc-macro specfic, and just export a trait. It should operate on tt::TokenStream, so a dep on ra_tt is required.

In other words, if we export a trait in ra_db, Could ra_proc_macrodepend on ra_db ? Or do we define a wrapper in ra_project_model to bridge them ?

[matklad]

Good question! Yeah, I think it would also be good if ra_proc_macro didn't depned on ra_db...

I think bridging them in project-model might work, but, if the expansion trait is really minimal and only depends on tt, we can just put it in the ra_tt?

[edwin0cheng]

I think it is plausible, I could imagine it looks like following:

enum ExpandError {
...
}

trait TokenExpander: Debug + Send + Sync + RefUnwindSafe {
    fn exander(&self, subtree: &Subtree, attrs: Option<&Subtree>) -> Result<Subtree, ExpandError>;
}

The trait bound here is needed in CrateData, as It will be stored in salsa.

[kiljacken]

Cargo interacting code looks reasonable to me 👍

Good job, this is some really cool work!

[edwin0cheng]

@matklad, I removed the dependency for mbe and refactoring a bit for salsa trait constraints. You could review it again. 👍

[matklad]

bors r+

[bors]

Build succeeded

• TypeScript
• Rust (windows-latest)
• Rust (macos-latest)
• Rust (ubuntu-latest)