How does call/cc work with the CPS transformation from "Lisp in Small Pieces"?

Question

The book Lisp in Small Pieces demonstrates a transformation from Scheme into continuation passing style (chapter 5.9.1, for those who have access to the book). The transformation represents continuations by lambda forms and call/cc is supposed to become equivalent to a simple (lambda (k f) (f k k)).

I do not understand how this can work because there is no distinction between application of functions and continuations.

Here is a version of the transformation stripped from everything except application (the full version can be found in this gist):

(define (cps e)
  (if (pair? e)
      (case (car e)
        ; ...
        (else (cps-application e)))
      (lambda (k) (k `,e))))

(define (cps-application e)
  (lambda (k)
    ((cps-terms e)
     (lambda (t*)
       (let ((d (gensym)))
         `(,(car t*) (lambda (,d) ,(k d))
                     . ,(cdr t*)))))))

(define (cps-terms e*)
  (if (pair? e*)
      (lambda (k)
        ((cps (car e*))
         (lambda (a)
           ((cps-terms (cdr e*))
            (lambda (a*)
              (k (cons a a*)))))))
      (lambda (k) (k '()))))

Now consider the CPS example from Wikipedia:

(define (f return)
  (return 2)
  3)

Above transformation would convert the application in the function body (return 2) to something like (return (lambda (g13) ...) 2). A continuation is passed as the first argument and the value 2 as the second argument. This would be fine if return was an ordinary function. However, return is supposed to be a continuation, which only takes a single argument.

I don't see how the pieces fit together. How can the transformation represent continuations as lambda forms but not give special consideration to their application?

Answer 1

I do not understand how this can work because there is no distinction between application of functions and continuations.

Implementing continuations without CPS requires approaches at the virtual machine level, such as using "spaghetti stacks": allocating lexical variables in heap-allocated frames that are subject to garbage collection. Capturing a continuation then just means obtaining an environment pointer which refers to a lexical frame in the spaghetti stack.

CPS builds a de facto spaghetti stack out of closures. A closure captures lexical bindings into an object with an indefinite lifetime. Under CPS, all closures capture the hidden variable k. That k serves the role of the parent frame pointer in the spaghetti stack; it chains the closures together.

Because the whole program is consistently CPS-transformed, there is a k parameter everywhere which points to a dynamically linked chain of closed-over environments that amounts to a de facto stack where execution can be restored.

The one missing piece of the puzzle is that CPS depends on tail calls. Tail calls ensure that we are not using the real stack; everything interesting is in the closed-over environments.

(However, even tail calls are not strictly required, as Henry Baker's approach, embodied in Chicken Scheme, teaches us. Our CPS-transformed code can use real calls that consume stack, but never return. Every once in a while we can move the reachable environment frames (and all contingent objects) from the stack into the heap, and rewind the stack pointer.)

Now consider the CPS example from Wikipedia:

Ah, but that's not a CPS example; that's an example of application code that uses continuations that are available somehow via call/cc.

It becomes CPS if either we transform it to CPS by hand, or use a compiler which does that mechanically.

However, return is supposed to be a continuation, which only takes a single argument.

Thus, return only takes a single argument because we're looking at application source code that hasn't been CPS-transformed.

The application-level continuations take one argument.

The CPS-implementation-level continuations will have the hidden k argument, like all functions.

The k parameter is analogous to a piece of machine context, like a stack or frame pointer. When using a conventional language, and call print("hello"), you don't ask, how come there is only one argument? Doesn't print have to receive the stack pointer so it knows where the parameters are? Of course when the print is compiled, the compiled code has a way of conveying that context from one function to another, invisible to the high level language.

In the case of CPS in Scheme, it's easy to get confused because the source and target language are both Scheme.