0

This maplit crate allows hashmap literals with => as a separator. I believe it's impossible to use macro_rules! to allow the : separator, but is it possible with a macro that processes a stream of tokens?

Is it impossible to add { key: value } style macros to the language, even in the compiler?

I suspect the problem is conflict with the : type operator, but I cannot construct an ambiguous example where it is impossible for the compiler to make a decision of which way to interpret :.

kmdreko
  • 42,554
  • 6
  • 57
  • 106
Test
  • 962
  • 9
  • 26
  • You can likely achieve what you want with a [function-like procedural macro](https://doc.rust-lang.org/reference/procedural-macros.html#function-like-procedural-macros). – EvilTak Mar 19 '22 at 19:14

2 Answers2

4

@kmdreko does a good job of explaining why you can't have an expression followed by a colon in a macro. However, you can work around that, by forcing the key to be a token tree rather than an expression:

macro_rules! hashmap{
    ( $($key:tt : $val:expr),* $(,)? ) =>{{
        #[allow(unused_mut)]
        let mut map = ::std::collections::HashMap::with_capacity(hashmap!(@count $($key),* ));
        $(
            #[allow(unused_parens)]
            let _ = map.insert($key, $val);
        )*
        map
    }};
    (@replace $_t:tt $e:expr ) => { $e };
    (@count $($t:tt)*) => { <[()]>::len(&[$( hashmap!(@replace $t ()) ),*]) }
}

playground

The drawback of taking this route, as opposed to a procedural macro, which probably could handle this pattern just fine, is that most complex expressions will have to be wrapped in parentheses or curly brackets. For example,

let map = hashmap!{ 2 + 2 : "foo" };

won't work, but

let map = hashmap!{ (2 + 2) : "foo" };

will.

Aiden4
  • 2,504
  • 1
  • 7
  • 24
  • Amazing. I proposed adding this to maplit. – Test Mar 19 '22 at 20:00
  • Thsi is truly incredible. I think it's the same way that the `json!()` macro in *serde-json* works for example, but this is still cool to see the explanation written out. – rv.kvetch Mar 19 '22 at 22:22
  • 1
    @rv.kvetch the `json!()` macro uses a much more complicated tt muncher that can parse `expr : expr`, and will promptly break whenever type ascriptions for expressions become a thing. – Aiden4 Mar 20 '22 at 00:24
  • Hi Aiden4 Amazing to your code, but I dont understand "@replace @count" in the code; could you explain more or give some links? – Henry Feb 24 '23 at 05:28
  • @Henry, they're just markers for internal use by the macro. It just lets me effectively have sub-macros, without actually needing to define them. Take a look [here](https://danielkeep.github.io/tlborm/book/pat-internal-rules.html) for more information (do note the book is a bit out of date, but probably still the best resource for understanding `macro_rules!` available. – Aiden4 Feb 25 '23 at 08:49
3

If you were to replace the => with a : in the maplit! macro you'd get this error:

error: `$key:expr` is followed by `:`, which is not allowed for `expr` fragments
 --> src/lib.rs:6:17
  |
6 |     ($($key:expr: $value:expr),*) => {
  |                 ^ not allowed after `expr` fragments
  |
  = note: allowed there are: `=>`, `,` or `;`

The specific problem is that the macro accepts any arbitrary expression as the key type. You could still have the { key: value } syntax, but only if the key is an ident (for example) instead of an expr.

You can read the Follow-set Ambiguity Restrictions section of Macros by Example in the Rust Reference:

The parser used by the macro system is reasonably powerful, but it is limited in order to prevent ambiguity in current or future versions of the language.

[...]

expr and stmt may only be followed by one of: =>, ,, or ;.

That's not to say its necessarily ambiguous, though it may get confusing to parse in examples with a lot of :s like MyEnum::Variant: ::std::collections::Vec::new(). Its designed like that because only the documented tokens are guaranteed to indicate the end of an expression and other tokens may become ambiguous in the future.

You'll see though, that this only limits how macro_rules! patterns are handled and would not affect a procedural macro since you have free reign to decide how the tokens are parsed.

I tried to create a simple procedural macro using syn to aid in parsing expressions, but it chokes on the Expr : Expr syntax because it attempts to eagerly parse the first expression as a type ascription expression which is a yet-to-be-completed feature. This is the kind of future expansion that the macro_rules! guards against.

Going with a token tree and using parenthesis for ambiguity in Aiden4's answer is the way to go if you really want : instead of =>. This still could be done with a procedural macro, but it would not be as readily available and would be ambiguous if type ascription expressions gets added to the language.

kmdreko
  • 42,554
  • 6
  • 57
  • 106