5

I've forked this question from a similar question I made in a comment I made to one of the many great answers I recieved. I was originally asking about AST macros, which mostly provoked very detailed and thoughtful responses from Lispers. Thanks.

Lazy Evaluation vs Macros

The question I made in a comment was whether project-specific DSLs are actually a good idea. Of course, this is completely subjective -- After all, when you are writing in a really expressive language, where do you draw the line between an expressive API and an actual DSL? For example, I think what most Rubyists call 'DSLs' are actually just well-designed APIs and nothing more.

Note that I say project-specific APIs. I don't think many will argue against using regular expressions or SQL where it makes sense to do so.

But despite this, I think we can all draw a vauge, hazy line between an API and a DSL. Of course they're both really APIs, but whatever.

On one extreme you have Lisp, where DSLs seem to be actively encouraged via macros. On the other you have the likes of Java where DSLs are pretty much impossible.

Proponents of DSLs would argue that they increase flexibility, expressiveness, and increase consistency (for example, a custom number object using the same operators as the language's own numbers).

Detractors would say that they can lead to sub-languages that nobody except the DSL writer knows, kills the point of having different programming languages in the first place, and leads to code nobody can understand because the way of interfacing with API is different.

I gotta say, I agree with both sides in many ways. Some Java APIs are just plain nasty due to the lack of expressiveness. Despite this, I can generally always work out what's going on without reading the documentation -- Which can't be said about custom DSLs in the slightest. Maybe DSL proponents argue that you should always read API documentation. I disagree, but I also digress.

But let's look at some of the big languages at the moment. C# and Java, namely. Neither of them really 'do' DSLs, yet they're massively popular. Is this precisely because they don't allow things like DSLs, allowing mediocre coders to churn out code that's still comprehensible?

Is the fact that DSLs allow mediocre coders to produce impenetrable garbage the reason why Lisp is not used as much as it should be, despite what a DSL can look like in the right hands?

Community
  • 1
  • 1
Louis
  • 2,442
  • 1
  • 18
  • 15
  • I think if you do a good job creating the DSL, it is going to transcend the individual project anyway :) – Affe Aug 13 '11 at 20:14
  • 1
    @Affe: Very true, lol. But to be fair there's only so many DSLs that can penetrate into every programmer's must-learn list like regular expressions and SQL has. – Louis Aug 13 '11 at 20:16
  • Who says you can't use Java to create a DSL? http://camel.apache.org/dsl.html – duffymo Aug 13 '11 at 20:41
  • @duffymo: You link proves my point: you can't do DSLs in Java, so they've had to implement the DSL with the combination of syntax-in-strings and chained method calls, it seems. A lot more clumsy than stuff you get in Haskell or Lisp, and considerably more error prone. – Louis Aug 13 '11 at 20:44
  • You can also check the Xtext project(http://www.eclipse.org/Xtext/) for creating DSLs in java. – Grigoris Grigoriadis Aug 13 '11 at 23:42
  • 4
    I just checked the Xtext Wikipedia page, and saw this statement: "Unlike standard parser generators, Xtext not only generates a parser, [...]". That implies it has to parse it, which in turn implies it ain't Java. Writing a whole parser in Java just to interpret a DSL shows that Java sucks at DSL construction. – Louis Aug 13 '11 at 23:59

2 Answers2

8

There are of course arguments in favor of DSLs and against them, and there's of course a vague line between "a library" or "an API" and "a DSL". That part you've covered well in the question, so I'll avoid those subjective points and focus on just the question of whether they're a liability.

A good project to consider for this is Racket which puts language construction as its main feature. It's easy to slap up a language for any definition of "a language": DSL or not, made up from near scratch via an interpreter or (more commonly) done via macros that define the new language and possibly a parser for a different syntax. As a result, the Racket source tree has a bunch of languages -- some of them have fundamentally different execution semantics. Some examples:

  • Lazy Racket is what you'd expect the name to mean,

  • Typed Racket is a statically typed language,

  • Scribble is a language for writing documentation and other prose,

  • Slideshow is a language for writing ... slideshows,

  • RackLog/DataLog are languages that are even more different in both semantics and syntax.

In fact, making up languages is so easy in Racket, that you easily slap up a language even if it's something that fits a very limited use, or even just a single one. For example, we have such "small languages" that are used to create our web pages, decide which files are included in distributed installers, and many many more. See this tutorial for a description of how to come up with a language.

It's true that there's a fine line between a useful DSL that many people can use and one that only one person uses -- but still, the kind of abstractions that you can get when you define a language rather than a library are substantial, to the point that it's a useful concept even when it's a "one-man's language". One hard problem in this are is considering interoperability -- Racket allows each module to be written in it's own language, which brings up issues of what happens when several of these modules are supposed to talk to each other. For example, how does evaluation proceeds when there's interaction of functions in the lazy language and in the default one; or how does the typed make sure that it can interact with the default untyped language and still get the usual benefits of a statically typed language.

Eli Barzilay
  • 29,301
  • 3
  • 67
  • 110
5

DSLs are widely used in Java. Just not 'internal' DSLs. Lisp has, unlike Java, several ways to alter the language syntax and semantics without writing a new external language. In Java, most DSLs are either external (often based on XML) or implemented with a pre-processor.

If you write a new programming language, also a domain-specific programming language, you need to:

  • specify it
  • document it
  • test it
  • make it robust
  • make it debuggable
  • make it efficient ... and more

Lisp is no magic bullet to do all that for you. What Lisp gives you is to include one or more DSLs directly into the language and it allows you to reuse or alter the Lisp facilities to implement parts of your new language.

Lisp had in its history a lot of languages implemented on top of it. Many of them were just research languages without too much effort to follow software engineering practices. Some of these languages had more effort put into them - for example because they were part of a product.

If you develop a language for your project, you need to make sure that you follow good software engineering practices and you have the resources to do so.

Rainer Joswig
  • 136,269
  • 10
  • 221
  • 346
  • 2
    Lisp certainly has a magic bullet: the whole point of macros is that they're a *local* kind of transformation which allows reusing the rest of the language, which is in sharp contrast to non-Lisp languages where you need to invest much more work. (See also [Felleisen's paper](http://www.ccs.neu.edu/scheme/pubs/scp91-felleisen.ps.gz) on this subject.) – Eli Barzilay Aug 13 '11 at 21:13
  • If a Java DSL has to be done via a preprocessor or XML then it is, by definition, not a Java DSL. But I get the general point of your post, but I'd just like to say that I wasn't starting a DSL-in-Lisp vs DSL-in-Java flamewar, just more on DSL viability itself, which your bullet points covered nicely. – Louis Aug 13 '11 at 21:15
  • @Louis, what is a 'Java DSL'? what is the definition of that? – Rainer Joswig Aug 14 '11 at 01:30
  • A DSL that's parsed directly via Javacc's parser at compile time. I.e., not parsed by a hand parser like many DSLs-in-strings, or by any parser-to-reflection libraries. Code generators don't count either. It has to be parsed by the language's own parser without any conversions beforehand. – Louis Aug 14 '11 at 02:23
  • That's **my** personal definition; I'm sure there are many Java coders out there who think that languages-in-strings or code generators are actually DSLs. They're just wrong :p – Louis Aug 14 '11 at 02:24
  • @Louis: which grammar does the parser use then? Java's grammar? Then it is Java, not another language. Another grammar? Then it is not Java. What about semantics? Java? Again? Then we have just Java again - not a new language. – Rainer Joswig Aug 14 '11 at 02:46
  • DSLs-in-strings use a runtime parser, thus it isn't Java's compile-time parser, thus it isn't Java. Any code that must go though parsing steps outside javacc's compile time isn't natively Java (although whether it's valid JVM bytecode is a whole different story). Javacc will stick to a well-defined grammar that you could probably find on Oracle's website somewhere. No doubt they'll have something like a BNF grammar for it. – Louis Aug 14 '11 at 02:54
  • @Louis: that does not make much sense. If you use Java and its grammar, then it is just Java. Not a different language. There is nothing like a Java DSL by your definition. – Rainer Joswig Aug 14 '11 at 03:38
  • But string-based DSLs **don't** use the Java grammar, which is why they need to have their meaning extracted by a runtime parser. Having a language **allow** a DSL to appear anywhere in its text -- in Java's case embedded within strings -- is **not** the same thing as having a language parse a DSL as a first-class citizen along with the rest of the code. Example of a real DSL in Haskell: http://augustss.blogspot.com/2009/02/more-basic-not-that-anybody-should-care.html. There's no way you can do something like that in Java using first-class syntax in the language. – Louis Aug 14 '11 at 03:54
  • @Louis: I have a feeling it would be possible to enable some compile-time "string interpretation" in Java by clever abuse of decorators. – Vatine Aug 14 '11 at 08:06