Programmatically constructing a Common Lisp function with defun

Question

I would like to define a function using defun, but not at top level. (The function name & body needs to be constructed from some user input.) The following illustrates the basic objective:

(let ((name 'fn1)
      (body "abc"))
  (eval `(defun ,name ()
           ,body))
  (push name *function-names*))

1) This works, but how can it be done without eval?

2) Currently, I'm compiling a number of such functions with

(loop for fn-name in *function-names* do
      (setf (symbol-function fn-name)
        (compile nil (symbol-function fn-name))))

but is there a better way using just the fn-name like `(compile ,fn-name) that avoids a SBCL compiler warning when the function body contains a recursive call to the function?

Answer 1

Doing it without DEFUN makes it a bit problematic. I think that's a part of ANSI Common Lisp, which is not so great.

If you look at implementations of Common Lisp, they expand a DEFUN form into implementation specific constructs. Often we'll see something like a named lambda.

SBCL:

(defun foo ()
  (foo))

->

(SB-INT:NAMED-LAMBDA FOO ()
  (BLOCK FOO (FOO)))

which evaluates to:

#<FUNCTION FOO {1003271FFB}>

Unfortunately, the named lambda is not a concept in ANSI CL.

To get that, you need to:

• construct the DEFUN form
• EVALuate it
• optionally COMPILE it

To get a similar effect we could create a file, compile and load it.

Alternatively we could try to not use DEFUN.

• we need to make sure that the global name is not a macro
• then we construct a lambda expression
• then we COMPILE that expression

Example: A recursive function:

(compile 'my-fn '(lambda () (my-fn))

Now we may still get a warning about an undefined function, while compiling the lambda. How can we provide a name? LABELS would do that:

(compile 'my-fn
         (lambda ()
           (labels ((my-fn ()
                      (my-fn)))
             (my-fn))))

LABELS will also set up the BLOCK, like DEFUN.

When we now call the global MY-FN, it will then call the local MY-FN, which then can call itself recursively.

Let's see if there are more alternative approaches...

• One could also use a normal LAMBDA with COMPILE and declare the function first. That might also suppress the compilation warning.

Answer 2

I am little confused as to "why" you are trying to do this with defun but if you want a user to be able to define their own functions and then call them, there are several ways you could this.

1) if you it to be evaluated like a normal defun function then you could just do something like

(loop for f in *my-functions-names* and
      for a in *my-functions-args* and
      for b in *my-functions-bodies*
  do (setf (symbol-function f) (compile `(lambda ,a ,b))))

then users could compile their functions at execution time and it would look like it had been there all the time.

2) create a hash table of lambdas and funcall (or apply) their functions:

(defparameter *user-functions* (make-hash-table))

(defun def-user-fun (f-name f-args f-body &optional (ns *user-functions*))
  (setf (gethash f-name ns) (compile nil `(lambda ,f-args ,f-body))))

(defun call-user-fun (f-name args &optional (ns *user-functions*))
  (funcall (gethash f-name *user-funcations) f-args))

This would allow users to define what ever functions they want which would be evaluated in the current environment.

3) if this is just to save you time with compiling your own code then you could just do what you did but with a defmacro and a loop statement.

Update

Since you need recursion then you can do something similar to what @rainerjoswig suggested with the (lambda (args..) (labels ((recurse ...)) (recurse ...))). Though it may not seem "nice" in an imperative way, this is idiomatic in Lisp. In fact, both the SBCL compiler and garbage collector are tuned specifically for this kind of recursion. If you want to know about Common Lisp optimizations you can depend on, you should read the standard.

Is (compile 'fname) == (setf (symbol-function 'fname) (compile nil (symbol-function 'fname)))? Yes and no. At least probably not in the way you think. It seems to me that you are confused about two things, both of which are part of the arcane machinery of lisp's "backend".

First, symbol-function is not an evaluator, it simply lets the lisp reader know that this symbol is a function symbol and should be accessed via the function environment and not the current lexical environment, that is, a function is accessed like a variable but in a different environment or namespace. I would wager that in the background how labels gets expanded is this function.

Second, compile also does not work that way. You can either pass it a symbol-function name which it accesses the stored function definition, compiles it, and then the old definition with the compiled definition, (compile 'my-func), you can pass it a lambda expression which it will compile, (compile nil (lambda (args...) body)), or, finally, you can pass both a name and a definition which will store the compiled function as that function variable.

So yes, in a sense it does expand to a setf but no to your specific setf because you should not do (compile nil (symbol-function 'fname)).

As for allowing recursion, it is simple enough to expand on @rainerjoswig by destructuring the form the general defun form like

(def-user-func count-down (val) 
  (if (> val 0)
    (progn (print val) (count-down (1- val)))
    nil))

with a macro like

(defmacro def-user-func (name args &body body)
  `(compile ',name (lambda (,@args)
                     (labels ((,name (,@args)
                                ,@body))
                        (,name ,@args)))))

Because of the way that lisp shadows variables names within let,labels, and other statements like it, this will work perfectly well. The "outside" world just sees a function while the function just sees its own labels function. When the ,@body expands, it will do so in the context of the labels and since we duplicated the names, it will work properly.