1

From the Syn documentation:

Syn operates on the token representation provided by the proc-macro2 crate from crates.io rather than using the compiler's built in proc-macro crate directly. This enables code using Syn to execute outside of the context of a procedural macro, such as in unit tests or build.rs

I am trying to enable unit testing for some Syn functions, however I can't get it to work no matter what I have tried. It does not work with the proc_macro2::TokenStream type, but it won't work with the proc_macro::TokenStream because we are not in a proc-macro context.

link to playground

use quote::quote;
use syn;

fn test() {
    // let stream: syn::export::TokenStream = quote!{fn foo() {};}.into(); // doesn't work
    let stream: proc_macro2::TokenStream = quote!{fn foo() {};}.into(); // doesn't work
    // let item = parse_macro_input!(stream as Item); // doesn't work
    let item = syn::parse(stream).unwrap();

}

fn main() {
    test();
}

Any help on how to test syn functions outside of the proc-macro context would be appreciated. I am aware of the trybuild crate, but I would like to be able to unit test the macro's functions first.

Peter Hall
  • 53,120
  • 14
  • 139
  • 204
Prgrm.celeritas
  • 311
  • 2
  • 12

1 Answers1

3

It does not work with the proc_macro2::TokenStream type, but it won't work with the proc_macro::TokenStream because we are not in a proc-macro context.

Yes, and that's the whole point! Crates that export procedural macros can't export anything else, but proc_macro can only be used in crates that export macros. This is the reason why proc_macro2 exists in the first place.

You need to use multiple crates in order to write tests for code that uses syn and proc_macro2:

  • Your public crate that declares the macros with #[proc_macro] etc., and does very little except convert a proc_macro::TokenStream into a proc_macro2::TokenStream and vice versa.
  • An "internal" crate, containing most of the actual code, which depends on proc_macro2 but not proc_macro. Your tests can go in here.

The error you are seeing is because syn::parse accepts a proc_macro::TokenStream. You can instead use syn::parse2, which is identical except that it accepts a proc_macro2::TokenStream.

Peter Hall
  • 53,120
  • 14
  • 139
  • 204
  • My code example above does not declare a macro or depend on `proc_macro`. What do my test need to look like to actually use the syn function? – Prgrm.celeritas Dec 12 '20 at 20:35
  • You can use `parse2` instead of `parse` to accept a `proc_macro2::TokenStream`. – Peter Hall Dec 12 '20 at 20:40
  • So it turns out that I don’t need a separate crate to write the tests, so long as I am using `syn::parse2` I can use it in the proc-macro crate. – Prgrm.celeritas Dec 13 '20 at 04:04