2

Recently I've been reading on how modern OS load executable programs and allocate memories for them. Unfortunately, I only have a computer science book in Russian as a reference, so, please, correct me if I am wrong, but it appears that modern OS have different sections in an executable program for data and actual processor commands. Furthermore, it is not possible to give the control to data section, i.e. one cannot store the command there. It is also not possible to change the processor command in executable (text) section.

Hence, the question: how does modern compiled CL (SBCL, Clozure-CL) does it? As far as I can understand it creates compiled FASL files and then loads them. But visible FASL files are created when a file is compiled. What happens when the function form is evaluated? Secondly, how does CL load them (at the level of machine/OS commands) so that a correct memory is allocated for them? Also, the old code must be off-loaded somehow.

PS. Of course, this is not an issue for interpreted languages. And things get even more complicated for JIT-compilers.

Rainer Joswig
  • 136,269
  • 10
  • 221
  • 346
mobiuseng
  • 2,326
  • 1
  • 16
  • 30
  • 2
    I can’t give a full answer but the rough steps would go: source code -> reading -> macroexpansion -> compiling -> linking -> loading -> running. A typical way to deal with top level forms is to compile them into some anonymous top level function which is called to finish the loading. Linking is needed for eg GC and finding symbols. In an os the actual separation of what can and can’t be executed is a property of memory pages. This is automatically set appropriately when loading a binary by the OS. JIT compiled languages can use eg `mprotect` to do this. – Dan Robertson Jun 28 '18 at 02:24
  • What do you mean when you say things are more complicated for JIT compilers? How is Common Lisp (fasl or not) compilation to native code different from JIT? – Dan Robertson Jul 20 '18 at 19:25
  • @DanRobertson With JIT things exist in intermediate form (byte-code) and compiled. On the top of it, JVM, for example, monitors the code execution and tries to optimize the code "on the fly" (how successfully is a different question). CL route is a bit more static. – mobiuseng Jul 21 '18 at 11:51
  • surely one could consider fasl such a bytecode. At least for some implementations? And aren’t JavaScript implementations considered JIT compilers even though they input source code? I don’t know if any tracing CL compiler but a tracing JIT is different from a JIT in general – Dan Robertson Jul 21 '18 at 11:56
  • @DanRobertson Yes, indeed, when I said "JIT" in this context I meant "tracing JIT". As for FASL as byte-code: to my knowledge SBCL and CCL produce native code, CLISP doesn't have JIT. Not sure about ECL and I don't have access to other compilers. – mobiuseng Jul 21 '18 at 12:06
  • AFAIK the JVM took the idea of bytecode from Lips's FASL the difference is that the JVM handles this on its own, in an automatic way and as user of the language you are not expected to use this. While Common Lisp Is manual and is open to you. – kisai Nov 11 '19 at 03:06

1 Answers1

1

Maybe this clarifies things: Google Groups

Unlike C you are not compiling your programs, and then executing them. Instead you are working inside the lisp environment. When you have loaded your file into sbcl as above, you can just type

(hello)

And your code will be executed, and the code you run is compiled, and not interpreted. In SBCL you don't need to explicitly compile first, since all code you type or load into sbcl is compiled on the fly. In fact there is no interpreter in sbcl. In other lisp systems you may need to compile explicitly to get the code compiled though.

The fasl file is not executable by itself, it must be used together with sbcl.

hr0m
  • 2,643
  • 5
  • 28
  • 39
  • It doesn't unfortunately. This is still a big picture. I'm interested in mechanics. FASL file needs to be loaded in such a way that instruction pointer can reach it. Just loading a file will load it into data section — you can't get the control pointer there. – mobiuseng Jul 21 '18 at 11:54
  • @mobiuseng There is no such difference between text and data once something is in main memory. Whether something may be executed is decided by permission bits in the page table. – Dan Robertson Jul 21 '18 at 13:12