Emacs: the code in the body of a defun or defmacro cannot refer to surrounding lexical variables?

Question

Update 2013 May: As of GNU Emacs 24.3.1, (let .. (defun..)) bytecompiles just fine without warning and the bytecompiled code works the same as not-compiled code. Just don't forget to add the file variable lexical-binding: t to the file to be bytecompiled. Workarounds at the end of this question is now not necessary.

Lexical Binding - Emacs Lisp Manual has this paragraph:

Note that functions like symbol-value, boundp, and set only retrieve or modify a variable's dynamic binding (i.e. the contents of its symbol's value cell). Also, the code in the body of a defun or defmacro cannot refer to surrounding lexical variables.

I am not sure if I am getting the meaning of the second sentence right. In the following code which should be run in lexical binding mode, the code in the body of a defun is successfully referring to the lexical binding value of the name n.

(let ((n 0))
  (defun my-counter ()
    (incf n)))

(my-counter) ;; 1
(my-counter) ;; 2

Is the sentence simply saying that (let .. (defun ..)) is a bad practice?

Workarounds:

;; -*- lexical-binding: t -*-

;; a way to define the counter function without byte-compile error or warning

(defvar my--counter-func
  (let ((n 0))
    (lambda ()
      (setq n (1+ n)))))

(defun my-counter ()
  (funcall my--counter-func))

;; another way to define the counter function, again without byte-compile error or warning

(fset 'my-another-counter
      (let ((n 0))
        (lambda ()
          (setq n (1+ n)))))

And here's the code for testing the above code:

;; run:
;; emacs -q --load path-to-the-el-file-of-this-code.el

(load "path-to-file-defining-my-counter.elc") ;; loading the ELC file to test if byte-compiled code runs as expected.

(print (my-counter)) ;; 1
(print (my-counter)) ;; 2

(print (my-another-counter)) ;; 1
(print (my-another-counter)) ;; 2

dkim · Accepted Answer · 2012-08-20T06:13:57.487

The code does not byte-compile well at least in Emacs 24.1.1. I saved the following code in the foo.el file, which uses setq in place of incf in order to avoid any possible effects by the cl library:

;; -*- lexical-binding: t -*-

(let ((n 0))
  (defun my-counter ()
    (setq n (1+ n))))

When I tried to byte-compile it (M-x byte-compile-filefoo.el), I got the following warning messages:

foo.el:3:1:Warning: Function my-counter will ignore its context (n)
foo.el:3:1:Warning: Unused lexical variable `n'
foo.el:5:11:Warning: reference to free variable `n'
foo.el:5:17:Warning: assignment to free variable `n'

All of the messages are indicating that the code in the body of the defun construct cannot refer to the surrounding lexical variable n as the manual claims.

Actually, when I loaded the byte-compiled code (M-x load-filefoo.elc) and evaluted the (my-counter) form, I got the following erorr:

Debugger entered--Lisp error: (void-variable n)
  ...

Unfortunately, I'm not sure why the code appears to work when evaluated in the form of source code.

I don't know how up to date this comment is on the EmacsWiki but it seems to suggest that there are problems with lexical scope when compiling: "There is compiler support for byte-compiling code to use lexical binding, but it is unfinished: Basic code works great, but the things that need more compiler analysis are unfinished or somewhat flaky (e.g., lexical closures work, but there are certainly still problems in that area)." http://emacswiki.org/emacs/DynamicBindingVsLexicalBinding — justinhj, Aug 20 '12 at 06:35
@justinhj: I don't know what that comment refers to, indeed. AFAIK the byte-compiler in 24.1 fully supports lexical scoping. — Stefan, Aug 22 '12 at 15:37

score 1 · Answer 2 · answered Aug 22 '12 at 15:38

1

As I replied on gnu.emacs.help, you can use (defalias 'foo (lambda ...)) to work around that limitation. And that limitation is lifted in Emacs's development code.

answered Aug 22 '12 at 15:38

Stefan

27,908
4
53
82

justinhj · Answer 3 · 2012-08-27T18:39:19.493

It's perfectly fine to refer to variables in lexical scope(*) from within a defun, just as you are doing above, and just as the "my-ticker" example on that manual page does as well.

Either I am missing something the line in the manual that says:

the code in the body of a defun or defmacro cannot refer to surrounding lexical variables.

should say something more like:

code in the body of a defun can only access lexical variables if they are defined within the same lexical scope.

NOTE: There are comments in the other answers about problems byte-compiling this kind of code. Those should be fixed in the latest emacs. I've verified in v24.2.50.1 that this byte compiles and loads correctly.

Emacs: the code in the body of a defun or defmacro cannot refer to surrounding lexical variables?

3 Answers3