2

I understand that dynamic languages (Ruby, Clojure, Groovy) provide metaprogramming support, but how about Scala?

Is there a use case for metaprogramming in Scala for DSL creation?

Caveat - I don't fully understand metaprogramming, but consider it potentially helpful for handling creating a new DSL grammar/rule at run-time (which may/may not be correct).

I've read this useful question, but I'm more interested in why(if there is one) to use metaprogramming in Scala.

Community
  • 1
  • 1
Kevin Meredith
  • 41,036
  • 63
  • 209
  • 384

1 Answers1

3

Metaprogramming in Scala now mostly revolves around Scala macros, a relatively fresh and still experimental compile-time metaprogramming facility.

Despite their relative immaturity and complexity (they're not easy to write), macros are already used in multitude of projects, since they look very promising for many purposes. See for example this video: What Are Macros Good For

ghik
  • 10,706
  • 1
  • 37
  • 50
  • With Scala's rich internal DSL and powerful parser combinators, why would I need macros? I watched that video, but did not understand much – Kevin Meredith Jan 15 '14 at 02:48
  • Some reasons include: 1) better performance and less boilerplate thanks to compile-time code generation (e.g. parser combinators are slow, but thanks to macros they can be made to be fast), 2) more powerful validation than the types currently provide, 3) direct access to the structure of DSL programs that users write instead of having to route the users through a number of specially arranged methods to capture their intent (e.g. see the language-integrated queries example). – Eugene Burmako Jan 15 '14 at 06:48
  • @KevinMeredith, an ad-hoc combinators-based runtime DSL interpreter is almost always harder to implement and debug than a fully static, staged, well-designed eDSL compiler. Interpretation is always worse than a compilation in nearly all possible ways. Unfortunately, Scala macros are very limited, unlike Lisp macros. – SK-logic Jan 15 '14 at 16:33
  • @SK-logic, conceptually, my understanding is that compile-time macros beat run-time rules' evaluation since (1) compile-time > run-time. However, is the current challenge with Scala macros summed up by **ghik**: `... their relative immaturity and complexity (they're not easy to write)`? As a non-beginner, non-expert Scala developer, I find parser combinators to be clear for implementing simple rules. – Kevin Meredith Jan 15 '14 at 19:51
  • Also, *static* macro-based DSLs have to be re-compiled to add another rule to the DSL, right? `Dynamic` DSLs (like Parser Combinators in Scala) can parse rules on the fly. – Kevin Meredith Jan 15 '14 at 21:29
  • @KevinMeredith It is also important to note that because Scala is statically typed, macros in Scala give you access to a huge amount of very detailed data that only exists in compile time. Statically typed ASTs simply contain much, much more information than untyped ones. – ghik Jan 15 '14 at 21:31
  • @KevinMeredith, as I said, Scala macros are ill-designed, unfortunately. There are some well known techniques that allows to extend, say, parsing rules dynamically using statically compiled eDSLs, but I'm not sure yet how to fit such things into limited Scala macro system. – SK-logic Jan 16 '14 at 00:02
  • 1
    @SK-logic Could you elaborate on why Scala macros are ill-designed? – Eugene Burmako Jan 16 '14 at 19:31
  • @EugeneBurmako, macro arguments are limited to the valid Scala AST (forcing eDSL designers to follow the Scala syntax conventions). Macros cannot be used in the same module they're defined in (which limits staging, the most important tool for simplifying metaprogramming). Separating macro definition and implementation also limits stating (macros that produce macros). Lack of functionality for lifting the generated statements to the outer contexts, again, makes it nearly impossible to implement complex languages in a straightforward, staged way. And, finally, no way to modify the parser. – SK-logic Jan 16 '14 at 21:18
  • Scala macros have gotten a lot easier to use in 2.11 with the addition of quasiquotation. They're certainly not perfect, but writing out `q"x + y"` is **a _lot_ better** than `Apply(Select(Ident(TermName("x")), TermName("$plus")), List(Ident(TermName("y"))))` – KChaloux Jan 16 '14 at 23:00
  • @KChaloux, such functionality as quasiquotation should be easy to implement on top of the macro system itself. If it's not possible or not easy for some reason, then something is terribly wrong with the very design of the language. – SK-logic Jan 16 '14 at 23:27
  • @SK-logic The macros are still highly experimental. That's worth keeping in mind. But I'm not a language designer - maybe you should apply your expertise over at scalamacros.org and help them work their design out? – KChaloux Jan 16 '14 at 23:31
  • @SK-logic Some of the limitations are there because of implementation problems (separate compilation, macro defs vs macro impls) that we plan to eventually fix. Others (well-formed and well-typed arguments) are there, because we are experimenting with ways to merge Lisp and Scala traditions. My recent StrangeLoop talk elaborates on that: https://twitter.com/strangeloop_stl/status/423676760828280833. If you're interested, we could discuss the details at https://groups.google.com/forum/#!forum/scala-internals - I'd be very keen on learning more about your vision of macros. – Eugene Burmako Jan 17 '14 at 09:44
  • @EugeneBurmako, separate compilation should not necessarily be a problem (see some of the Scheme implementations for inspiration). I'm really glad that macro metaprogramming is pushing its way to the mainstream (and I consider Scala to be very close to the mainstream acceptance now), so it's extremely important to get it right. People will judge metaprogramming by its biggest ethos. You still can have a well-typed system without restricting the macro arguments to be Scala AST, but at a price of modifying the parser and having two stages of macro expansion: syntactic and semantic, both typed. – SK-logic Jan 17 '14 at 10:40
  • P.S. I believe that staging is much, much more important for a safe, clean and practical metaprogramming than a mere type safety. It's very hard to do any even marginally non-trivial macro transform in a single step, but it becomes totally trivial if the expansion is done in small and simple steps chained together. So my usual metric for any meta-language is to measure how easy it is to chain macro expansions in it. – SK-logic Jan 17 '14 at 10:52
  • @SK-logic Yes, there is the responsibility of getting this right for great good, which is why Scala macros are still experimental, so that we have freedom to figure out the best course of action. Let's continue our chat on our mailing list (https://groups.google.com/forum/#!forum/scala-internals)? – Eugene Burmako Jan 17 '14 at 10:53