Why do list-tail raise-exception? Why do we haven't closure property with respect to cdr?

Question

If you evaluate (list-tail '(1 2) 3) at guile scheme. You will get an exception. It would be smarter to have an '() as answer. Overall why do we haven't closure property with respect to cdr combinator? What complications may arise?

Examples to make my point clearer Now (cdr (cdr (cdr '(1 2))) -> raise-exception Should be (cdr (cdr (cdr ... (cdr '(1 2))...))) -> ()

Then we would automatically have properly working list-tail

(define (list-tail list n) 
  (if (= n 0)
      list
      (list-tail (cdr list) (- n 1)))

Group-by then could be written elegantly and exceptionless

(define (group-by list-arg n)
  (if (null? list-arg)
      '()
      (cons (list-head n) (list-tail n))))

Answer 1

The historic answer is that:

• Originally, the Lisp 1 and 1.5 languages created by John MacCarthy did not allow (CDR NIL). The CDR function required a cons cell argument.

• The idea that it would be convenient for (CDR NIL) to just return NIL came from a dialect called Interlisp (but may have been present elsewhere).

• In the 1960's, there was another major dialect of Lisp called MacLisp (two decades before the Apple Mac, unrelated).

According to The Evolution of Lisp by Peter Gabriel and Guy Steele, some MacLisp people held a pow-wow with Interlisp people in 1974:

In 1974, about a dozen persons attended a meeting at MIT between the MacLisp and Interlisp implementors, including Warren Teitelman, Alice Hartley, Jon L White, Jeff Golden, and Guy Steele. There was some hope of finding substantial common ground, but the meeting actually served to illustrate the great chasm separating the two groups, in everything from implementation details to overall design philosophy. [...] In the end only a trivial exchange of features resulted from “the great MacLisp/Interlisp summit”: MacLisp adopted from Interlisp the behavior (CAR NIL) → NIL and (CDR NIL) → NIL, and Interlisp adopted the concept of a read table.

Both Interlisp and MacLisp are ancestral dialects to Common Lisp, which also has the forgiving car and cdr.

Further remarks are made in the above paper on this matter, begining with:

The adoption of the Interlisp treatment of NIL was not received with universal warmth.

You can see from this fifty, sixty years ago, Lisp people were already divided into camps, and didn't agree on everything. Whether the car of an empty list should just yield the empty list, or error out is a very old issue.

Ashwin Ram, presently director of AI at Google, put in his own opinion in favor of forgiving cdr in 1986, when he composed this poem.

It still remains a divisive issue that is a matter of opinion.

It is undeniable that the flexible car, cdr and their derivatives can help you "code golf" list processing code.

It's also true that such code-golfed code sometimes handles only happy cases without error checking, which can cause problems in some circumstances.

For instance, some list that is assumed to always have three items is subject to (caddr list) to get the third item. But, due to some bug, it has only two. Now the code just ran off with the nil value, which may cause a problem somewhwere else. For instance, suppose the value is expected to be a string, and in some totally different function elsewhere, nil is passed to some API that needs a string and blows up. Now you're hunting through the code to discover where this nil came from.

People who write Lisp interpreters or compilers that rely on the forgiving destructuring performed by car and cdr end up producing something that accepts bad syntax silently.

For instance

(defun interpret-if (form env)
  (let ((test (car form))
        (then (cadr form))
        (else (caddr form)))
   (if (interpret-expr test env)
     (interpret-expr then env)
     (interpret-expr else env))))

This is actually a very nice example for discussing both sides of the issue.

On the one hand, the code is succinct, and nicely supports the optional else clause: the user of this interpreter can do:

(if (> x y)
  (print "x is greater than y"))

In interpret-if, the else variable will pull out a nil, and that will get handed off to (eval expr else env) where it just evaluates to nil, and everything is cool; the optionality of else was obtained from free thanks to caddr not complaining.

On the other hand, the interpreter doesn't diagnose this:

(if) ;; no arguments at all

or this:

(if (> x y))  ;; spec says "then" is required, but no error!

However, all these issues have nice solutions and ways of working that don't require tightening up the list accessor functions, so that we can resort to the succinct coding when we need to. For instance, the interpreter could use pattern matching of some kind, like Common Lisp's rudimentary destructuring-bind:

(defun interpret-if (form env)
  (destructuring-bind (test then &optional else) form
     (if (interpret-expr test env)
       (interpret-expr then env)
       (interpret-expr else env))))

destructuring-bind has strict checking. It generates code with car, caddr and other functions under the hood, but also error checking code. The list (1 2 3) will not be destructured by the pattern (a b).

You have to look at the entire language and how it is used and what else is in it.

Introducing forgiving car and cdr into Scheme might give you less mileage than you think. There is another issue, which is that the only Boolean false value in Scheme is #f. The empty list () in Scheme is not false.

Therefore, even if car is forgiving, code like this cannot work.

Suppose the third element of a list is always a number, or else it doesn't exist. In Lisp, we can do this to default to zero:

(or (third list) 0)

for that to work in Scheme in the default 0 case, (third list) would have to return the Boolean false value #f.

A plausible approach might be to have different default values for car and cdr:

(car ()) -> #f
(cdr ()) -> ()

However, that is rather arbitrary: it works in some circumstances, but fails in situations like:

;; if list has more than two items ...
(if (cddr list) ...)

If cddr returns () by default, then that is always true, and so the test is useless. Different defaulting for car and cdr would probably be more error prone than common defaulting.

In Lisp, the forgiving list accessors work in a synergistic way with the empty list being false, which is why once upon a time I was quite surprised to learn that the forgiving list accessors came in fairly late into the game.

Early Scheme was implemented as a project written in Lisp, and therefore to interoperate smoothly with the host language, it used the same convention: () being NIL being the empty list and false. This was eventually changed, and so if you're wishing to have that back, you're asking for Scheme to revert a many-decades-old decision which is next to impossible now.

Object-oriented programming weighs in on this also. The fact that (car nil) does something instead of failing is an instance of the Null Object Pattern, which is something useful and good. We can express this in the Common Lisp object system, in which it practically disappears:

Suppose we had a car function which blows up on non-conses. We could write a generic function kar which doesn't, like this:

;; generic fun
(defgeneric kar (obj))

;; method specialization for cons class: delegate to car.
(defmethod kar ((obj cons))
  (car obj))

;; specialization for null class:
(defmethod kar ((obj null))) ;; return nil

;; catch all specialization for any type
(defmethod kar ((obj t))
  :oops)

Test:

[1]> (kar nil)
NIL
[2]> (kar '(a . b))
A
[3]> (kar "string")
:OOPS

In CLOS, the class null is that class whose only instance is the object nil. When a method parameter specializes to null, that method is only eligible when the argument for that parameter nil.

The class t is the superclass of everything: the top of the type spindle. (There is a bottom of the type spindle also, the class named nil, which contains no instances and is a subclass of everything.)

Specializing methods on null lets us catch method calls with nil parameters. Thanks to CLOS multiple dispatch, these can be thus handled in any parameter position. Because of that, and null being a class, the Null Object Pattern disappears in CLOS.

If you're debating Lisp with OOP people, you can present (car nil) can be spoken about as being the Null Object pattern.

The inconvenience of explicit null handling is recognized in numerous newer programming languages.

A common feature nowadays is to have null safe object access. For instance

foo.bar

might blow up if foo is null. So the given language provides

foo?.bar

or similar, which will only dereference .bar if foo isn't nil, otherwise the expression yields nil.

When languages add foo?.bar, they do not throw away foo.bar, or make foo.bar behave like foo?.bar. Sometimes you want the error (foo being nil is a programming error you want to catch in testing) and sometimes you want the default.

Sometimes you want the default, so you can collapse multiple levels of default and catch an error:

if (foo?.bar?.xyzzy?.fun() == nil) {
  // we coudn't have fun(); handle it
  // this was because either foo was nil, or else bar was nil,
  // or else xyzzy was nil, or else fun() returned nil.
} else {
  // happy case
}

Answer 2

cdr is only allowed on pairs. When you reach the end of the list, the value is (), which is not a pair, so you get an error.

You can check for this in your list-tail procedure to allow it to be more permissive.

(define (list-tail list n)
  (if (or (= n 0) (not (pair? list)))
      list
      (list-tail (cdr list) (- n 1)))

Using (not (pair? list)) will also allow it to work for improper lists like (1 2 . 3). It will keep returning 3 for any n >= 2.

Answer 3

"You will get an exception."

This is a problem with many core libraries, and not just Scheme. Take Haskell's core library:

 tail [1] -- []
 tail []  -- error

 head [1] -- 1
 head []  -- error

As you know, the technical name for a function like this is a Partial Function. It is a function that doesn't work for some inputs, leading to errors.

So, yes, you can define your own version. One thing, though - what should be returned in the end condition? Should (list-tail '(1 2) 3) return () or should it return 0? If I'm trying to get a value to add to another number, then 0 would be appropriate. If I'm using cons to gather values then () would be appropriate. I guess that's why the function is left as partial.

"Right. I was interested why scheme was designed that way, without car/cdr closure property. Is it feature or just design flaw. It's more like scheme is less consistent than Common Lisp, rather strict."

Common Lisp returns NIL when it runs out of list:

(car '(1)) ; 1
(car '())  ; NIL
(cdr '(1)) ; 1
(cdr '())  ; NIL

In this case you would have to test for NIL, and if you wanted a zero instead make the replacement.

Answer 4

Why Scheme didn't have this is due to its minimalistic design. The report was so under-specified you could do pointer arithmetic and just let the program segfault since any faulty scheme code was deemed not scheme and pigs could fly. Later reports, like R7RS, requires much more error checking since it is required to signal errors in many situations where just undefined behavior would be OK in early reports.

With today's Scheme we can easily create car and cdr that does what you want:

#!r7rs

(define-library
  (sylwester pair-accessors)
  (export car cdr)
  (import (rename (scheme base) (car base:car) (cdr base:cdr))
          (except (scheme base) (car cdr)))
  (begin
    (define (car v)
      (if (pair? v)
          (base:car v)
          '()))
    (define (cdr v)
      (if (pair? v)
          (base:cdr v)
          '()))))

So in your library or program you just import (scheme) (or (scheme base)) without car and cdr and also import (sylwester pair-accessors) and you're in business. Alternatively you can make a (scheme base) or (scheme) that replaces all accessors with you own safe ones using a macro to produce them all.

The only thing you cannot do is inject your version of car/cdr into already defined libraries since that would require some late binding or monkey-patching, but that isn't supported by the language. I'm fascinated by these things and would love to make a OO-scheme where you can augment standard procedures with some CLOS-ish late binding where all core functions under the hood are indeed methods so that you can define your own objects and accessors and that standard libraries and user libraries created for normal pairs would just work out of the box for your new data structures that has pair like features.