0

I'm totally confused about clojure's reader. The input that the reader's read function gets passed can be source file or some ASCII text directly typed into the REPL. The reader then produces a data structure and passes it to the compiler so it could be evaluated.

What I don't understand is how that data structure 'looks like' for a compiler?

read-string function does the same as the reader's read function when it gets passed a string. read-string "(+ 1 2 3)") returns (+ 1 2 3). Does it mean then that (+ 1 2 3) is the exact representation which gets passed to the compiler as an internal data structure?

Why is the reader important as a separate function, idea, phase? Why doesn't the compiler serialize those reader forms and convert it to data structures internally?

Another question is: It is possible to write a program that would generate data structures directly and so they could then be directly passed to the compiler (without stepping through the reader stage), neither through a macro coded in a source file. How can try to do this?

Very nice explation: What are the tasks of the "reader" during Lisp interpretation?

Community
  • 1
  • 1
lavokad
  • 137
  • 2
  • 9

1 Answers1

3

The compiler does not receive characters as input, it receives structured Clojure data. The compiler resolves symbols to vars (by finding them in the appropriate namespace).

Let's look at what your read-string call actually returns:

user> (def input (read-string "(+ 1 2 3)"))
#'user/input
user> input
(+ 1 2 3)
user> (clojure.pprint/pprint (map (juxt identity type) input))
([+ clojure.lang.Symbol]
 [1 java.lang.Long]
 [2 java.lang.Long]
 [3 java.lang.Long])
nil

As you can see, there are no characters being sent to the compiler - the reader reduces everything to a primitive Clojure type (more or less the types supported by edn data), and the compiler turns this into runable code.

Regarding "generating these datastructures directly", this is exactly what macros do (though at least the macro invocation will be handled by the reader for a macro call).

user> (defmacro construct-addition [n] (list + 1 2 n))
#'user/construct-addition
user> (construct-addition 5)
8
user> (macroexpand '(construct-addition 5))
(#<core$_PLUS_ clojure.core$_PLUS_@190767cf> 1 2 5)
noisesmith
  • 20,076
  • 2
  • 41
  • 49
  • Thanks a lot. I don't get few things in your answer. Then the result of reader's read function execution is the this `([+ clojure.lang.Symbol] [1 java.lang.Long] [2 java.lang.Long] [3 java.lang.Long])` and it is then "passed" or being evaluated by the compiler? Is this the only representation that the reader generated and is now visible to compiler? Regarding macros, here is the link where you immediately can see the slide where two balloons appear (program and program(macro)), which both go the compiler balloon. https://www.youtube.com/watch?v=P76Vbsk_3J0#t=3580 Then are they same? – lavokad May 27 '14 at 21:45
  • No, the result of the read function is equivalent to `(list '+ 1 2 3)`, I used juxt to show the printed form and the type of each element of that list. The compiler looks for macros in its input, and expands them, and then expands macros in that result etc. until no more macros are left to be evaluated. The fundamental process is similar, but macros give access to that compile time environment (so they can change evaluation rules by preventing some input from being evaluated for example). – noisesmith May 27 '14 at 22:11
  • it sees the list `(+ 1 2 3)` which is equivalent to the result of `(list '+ 1 2 3)`, the types are produced by the reader (see the link regarding edn for more details on what types those can be). – noisesmith May 27 '14 at 22:39
  • Hi @noisesmith, I modified a bit my question and would like to ask you to see once more if you can find any other helping explanation. This post is about exactly what i'm wandering and the discussion is pretty amazing on it. http://calculist.org/blog/2012/04/17/homoiconicity-isnt-the-point/ Thanks – lavokad May 29 '14 at 20:49
  • Without the reader, you would have to either use the clojure compiler from java (generating the code from java code and data) or make your own parsing program that generates the right jvm bytecode in some other manner. You would need to write your own parser (or a serializer / deserializer for the parsed form). – noisesmith May 29 '14 at 20:53
  • further, `read` is that function that converts the ascii to the internal (list) structure, and the compiler turns that into the runnable bytecode – noisesmith May 29 '14 at 20:55