What fragment specifiers (metavariable types) should be used in macro-by-example for a module path + type name?

Question

I'm finding myself write code like this:

// Of course I could add a use statement, but there's still undesireable duplication.
// in the real code theres anywhere from 3 to 10 items in the tuple,
// but that would just clog up this example
pub fn cols() -> (crate::foo::bar::A, crate::foo::bar::B) {
    (crate::foo::bar::A, crate::foo::bar::B)
}

Many times over. I tried to create a macro to spit out this function for me:

macro_rules! impl_cols {
    ( $namespace:path, $($col_name:ty,)*) => {
        pub fn cols() -> ( $( $namespace::$col_name, )* ) {
            ( $( $namespace::$col_name, )* )
        }
    }
}

but no matter what fragment specifier I pick for the metavariables (path,ident,ty,tt, combinations thereof) it errors. What's the magic incantation to get this to work?

The types A and B look something like this:

// In "foo.rs"

pub mod bar {
    pub struct A;
    pub struct B;
}

An (erroring) example on the rust playground.

There is of course an XY problem here: I'm using the diesel crate and have structs with Queryable derive impl on them, and I want to be able to .select() the correct columns to fill a given struct. While there probably are other solutions, I still want to understand why the macro I wrote doesn't work, and what would work if anything.

Chayim Friedman · Accepted Answer · 2021-11-08T23:02:24.900

You can do that with tt-munching:

macro_rules! impl_cols {
    (@build-tuple
        ( $($types:path,)* )
        ( $($ns:ident)::* )
        ( $col_name:ident, $($rest:tt)* )
    ) => {
        impl_cols! { @build-tuple
            (
                $($types,)* 
                $($ns::)* $col_name,
            )
            ( $($ns)::* )
            ( $($rest)* )
        }
    };
    // Empty case
    (@build-tuple
        ( $($types:path,)* )
        ( $($ns:ident)::* )
        ( )
    ) => {
        ( $($types,)* )
    };
    (
        $($ns:ident)::*,
        $($col_name:ident,)*
    ) => {
        pub fn cols() -> impl_cols! { @build-tuple
            ( )
            ( $($ns)::* )
            ( $($col_name,)* )
        } {
            impl_cols! { @build-tuple
                ( )
                ( $($ns)::* )
                ( $($col_name,)* )
            }
        }
    };
}

Playground.

Edit:

The reason why simple matching doesn't work is that you cannot concatenate paths in Rust macro (without the help of a procedural macro). This is because, quoting the reference (https://doc.rust-lang.org/reference/macros-by-example.html#transcribing):

When forwarding a matched fragment to another macro-by-example, matchers in the second macro will see an opaque AST of the fragment type. The second macro can't use literal tokens to match the fragments in the matcher, only a fragment specifier of the same type. The ident, lifetime, and tt fragment types are an exception, and can be matched by literal tokens.

This is true not only when forwarding to another macro, but also when forwarding to the compiler.

You want to build two different AST fragments: a Type for the return type (each type in the tuple), and an Expression for the body. That is, crate::foo::bar::A (for instance) fulfill two roles: in the return type it is a type (specifically TypePath), and in the body it is an expression (specifically PathExpression).

If we look at the definition of both (TypePath and PathExpression), we see they're essentially equal to the following (ignoring irrelevant parts like generics and function paths):

Path :
::^? PathIdentSegment (:: PathIdentSegment)^*

If you are not familiar with EBNF notation, this means a list of identifiers (:ident in macro_rules!), separated by ::s.

So, when you're doing something like:

macro_rules! concat_ns {
    ($ns:path, $type:ident) => {
        fn my_fn() -> $ns :: $type { todo!() }
    };
}
concat_ns!(crate::foo::bar, A)

Your macro invocation builds an AST similar to:

MacroInvocation
  ...
    Path
      PathIdentSegment `crate`
      PathIdentSegment `foo`
      PathIdentSegment `bar`
    COMMA
    IDENTIFIER `A`

Your macro wants to build an AST similar to:

Function
  ...
  FunctionReturnType
    Type
      Path
        <Insert metavariable $ns here>
        <Insert metavariable $type here>

Which gives you:

Function
  ...
  FunctionReturnType
    Type
      Path
        Path
          PathIdentSegment `crate`
          PathIdentSegment `foo`
          PathIdentSegment `bar`
        PathIdentSegment `A`

But this is an invalid AST, since Path can contain only PathIdentSegment and not other Paths! (Note: this is not the exact process, but it is more-or-less the same).

Now you also understand why the tt-munching solution works: there, we never create a Path node, and just keep the raw identifiers. We can concatenate raw identifiers and create a single Path from them (this is generally the reason tt-munchers are used: when we need to not use Rust macro's syntax fragment capturing abilites because we want to restore them after).

Do you know why the `path` token type doesn't work? – Shelvacu Nov 08 '21 at 18:50 — Shelvacu, Nov 08 '21 at 18:50
@Shelvacu Added a explanation. – Chayim Friedman Nov 08 '21 at 23:02 — Chayim Friedman, Nov 08 '21 at 23:02

What fragment specifiers (metavariable types) should be used in macro-by-example for a module path + type name?

1 Answers1

Linked