16

Every Common Lisp programmer knows that macros are a powerful tool. Common Lisp macros have been used, among other things, to add object orientation on top of Lisp without changing the language specification; read-macros are another construct with mind bending capabilities.

Another program which allows meta-programming is Forth. Forth does it in a slightly different manner, using 'words' and 'generate words'.

I would like to know, from someone who dabbled in both languages, if common lisp macros and forth constructs are comparable in breadth/power: is there something you can do with the former which you can't do with the latter? Or vice-versa?

Of course, I am not talking about Turing-completeness of the two languages: I am talking about metaprogramming capabilities. C is Turing-complete but only a fool would state that C macros are comparable in power to Common Lisp ones.

user3750103
  • 193
  • 1
  • 5
  • 3
    This is much more specific than the previous version, [Lisp and Forth macros \[on hold\]](http://stackoverflow.com/q/24272856/1281433), which is good. Of course, now *that* question has reopen votes, and this has a close vote, so there's some confusion about which of these should actually stay open. It's much clearer what you're looking for in this question, but it's still probably off topic, because ["comparison question are a poor fit for \[Stack Overflow\], because there are no bounds to the answers which can be posted to them."](http://meta.stackoverflow.com/q/251328/1281433). – Joshua Taylor Jun 18 '14 at 13:42
  • 5
    The close reason **too broad** applies here: "There are either *too many possible answers*, or good answers would be too long for this format." This is not a **bad** question at all; it's just not a particularly good fit for Stack Overflow. This question could be great in comp.lang.lisp, for instance. – Joshua Taylor Jun 18 '14 at 13:44
  • 1
    The particular subquestion, "is there something you can do with the former which you can't do with the latter? Or vice-versa?" is probably the most specific here. It still might admit too many answers, but if there's something that one can do that the other can't, it's probably possible to give a relatively canonical answer. – Joshua Taylor Jun 18 '14 at 13:45
  • 1
    Against closing for **too broad**. Example answer: "in `LANG1` you can't implement `foo`, which has been implemented in `LANG2`, because you lack `bar`, which `LANG1` has and `LANG2` has not." In the [Why are some questions marked "on hold"?](http://stackoverflow.com/help/closed-questions) there is no mention of *"too many answer"* as a reason questions should be closed. – user3750103 Jun 18 '14 at 15:17
  • 2
    Actually, it's right there in the page that you linked to: "**too broad** - if your question could be answered by an entire book, or has many valid answers, it's probably too broad for our format *There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow down the answer set or to isolate an issue that can be answered in a few paragraphs.*" – Joshua Taylor Jun 18 '14 at 20:41
  • 1
    'Common Lisp macros have been used, among other things, to add object orientation on top of Lisp without changing the language specification'. Sounds wrong. CLOS for example is much more than macros and the specification of various language facilities was changed for it. – Rainer Joswig Oct 12 '14 at 07:29
  • There is a very good answer to this question in a great book by Doug Hoyte, [Let Over Lambda—50 Years of Lisp](https://letoverlambda.com/index.cl/toc), most of which is online. He only leaves out the last two chapters, one about performance and the other about implementing a Forth on Lisp, where you find your best answer. I bought the book, and am glad I did. Both Forth and Lisp are about power because you can implement anything on top of either, including any kind of macros (code that writes code). This book is worth getting for how it reveals and teaches macros, and answers your question. – MicroservicesOnDDD Dec 06 '19 at 15:31

3 Answers3

9

In my view, Common Lisp macros are similar to Forth immediate words. (Actually, they are most similar to Lisp reader macros.)

  • They are both procedural macros, i.e. can use full power of the language.

  • They both have access to the source code input.

  • They both can output anything expressible in the language. (E.g. an object-oriented extension in Lisp, or basic control flow constructs in Forth.)

The main difference, perhaps, would be that Forth "macro" input are character strings, while Lisp macros operate on a parse tree.

Lars Brinkhoff
  • 13,542
  • 2
  • 28
  • 48
  • 4
    Lisp macros works on S-expressions, which I suppose could look like parse trees to non-Lispers, but still. I mention this distinction because in pretty much any Lisp dialect, the reading phase comes before the compilation phase, and so Lispers see code _as_ data when writing macros. Yay homoiconicity! (You clearly know all of this since [common-lisp] is a top tag for you, but I feel other readers here should understand the difference too.) – C. K. Young Jun 18 '14 at 10:43
  • Forth immediate words can also operate at the word level instead of input stream level - it mainly depends on what customization points an implementation provides. They can also leave the input stream untouched. Furthermore, with `[` and `]` to switch the compilation off and on, respectively, you can define "immediate words" on the fly, i.e. get compile-time constants etc. That is, incidentally, stupidly hard in Python, so for me Python and FORTH + LISP are in two different classes. Python is more like C++ would be had it had an exposed constexpr parser API... – Kuba hasn't forgotten Monica Jul 01 '21 at 17:28
9

I'm a Forth implementor and have broad experience in Forth, half a dozen implementations, a few hundred projecteuler problems. I also did a small object oriented extension (small, i.e. a dozen lines or so). My experience in LISP is much more at the course level, but I think it is fair to say that the meta programming facilities of LISP are more systematic, and in practice more powerful. On the other hand Forth shines in transforming the language, and if your goal is to yield something different, even unrecognizable as Forth, Forth is the choice.

It is relatively easy to build abstractions on top of abstractions in LISP. In Forth if I want to define a matrix on a linear space defined by objects with non-trivial definitions .. I switch to Python.

The reason is also evident, at least to me. The creators of LISP where mathematicians, while Chuck Moore is the ultimate example of a practical programmer, never wasting his time on a theoretical problem.

This extends to this question in the following way. A lisp macro has a structure in the context of Lisp and at least suggests a meaning that is in line with general lisp principles. A Forth immediate word is just another program and can mean and do anything, including making a dogs dinner out of it.

Albert van der Horst
  • 854
  • 6
  • 19
  • Having done quite a bit of work in LISP, Python and FORTH, I'd say that the FORTH immediate words have perhaps the most power and enable highly expressive domain-specific languages. Try patching into the parser input stream in Python... even in LISP, it's not something for the faint of heart. Incidentally, a matrix of complex objects performs better in FORTH, because you only pay the price for the object properties you actually need - that's assuming your implementation doesn't just copy Python. In Python, the `object`'s virtual method table is enormous and a serious scaling bottleneck. – Kuba hasn't forgotten Monica Jul 01 '21 at 17:18
  • In FORTH, immediate words are a compiler customization point that's akin to a single virtual method. Mathematically, their semantics are quite definite and not hard to formalize conclusively - except that the details vary with each implementation, of course, since some implementations effectively provide multiple such virtual methods to override. But for any particular implementation, it's not hard to formalize what the immediate words are. – Kuba hasn't forgotten Monica Jul 01 '21 at 17:21
3

Sorry if the discussion bellow may seem a little bit vague, but there is too much to say.

I have only theoretical knowledge of Lisp, not hands on.

On the other side, Forth could have (but normal Forth hasn't) a complete metaprogramming inside the language. But metaprogramming is contemplated and possible but without a coherent syntax. I think the same happens in Lisp.

I have implemented a very clean solution with includes that possibility in a small paradigm of Forth like language. In order to have metaprogramming we should be able to refer to what we write. So when we write a program to be executed immediatelly as:

bread eat

we should be able too to refer to the same phrase with the intent instead of executing it to keep it for later reference. That could be done writing e.g.

{ bread eat }

The phrase above could have as consequence to leave the created object on the stack. But as we created a new word as { and } we have the right to refer to that too.

So, we could like to refer to:

{ bread

How could we refer to that ? A tentative syntax is: {{ { bread }}.

If we give the name XXX to the previous phrase we could write the initial phrase as:

XXX eat }

and the phrase above should work correctly in leaving on the stack the

{ bread eat }

As I don't know if what I say is exactly what are you looking for let it suffice to say that by the above reasoning and their implementation inside Forth each word gets an execution level and that defines which is the level of metaprogramming of the word.

Obviously, we have the first infinity level and every successive level. So execution levels are based on the mathematical infinity.

I have implemented the above inside a kind of Forth and at the first level (bellow infinity) everything works smoothly. So for example a am able to change the syntax of:

{ bread eat } { tomatos eat } x 3 = if

into:

{ bread eat | tomatos eat } x 3 = if

That is done by defining | as } { as bellow:

{{ } { }} "|" define

or if you like it better as: 

2{ } { 2} "|" define

The method above takes inside the language with a correct syntax the meta language and makes it language.

So in my opinion, both Lisp and Forth have the possibility of metaprogramming, but both lack that possibility as an integrating part of the language.

I have more details on that on napl.wikispaces.com.

Mark Karpov
  • 7,499
  • 2
  • 27
  • 62
George Kourtis
  • 2,381
  • 3
  • 18
  • 28