Why is this global variable only able to be set twice, and no more?

Question

I was trying to make a simple parser in elisp, and I encountered a problem where I defvar a global, then later I setq a new value into it. This works the first time. However, later setq's fail every time.

The following code is a simplication of the problem:

(defvar buf '(BUF))
(defvar head nil)

(defun pparse (seq)

  (defun status ()
    (princ (format "Parse: %40s || %-20s\n"
           (prin1-to-string seq) (prin1-to-string buf))))

  (while seq
    (status)
    (setq head (car seq))
    (setq seq (cdr seq))
    (cond ((equal "x" head)
       (nconc buf (list head)))
      ((equal "," head)
       (setq buf '(BUF))
       ;;(setcdr buf nil) <- fixes it but doesn't answer my question
      )))

  (status))

(pparse '("x" "," "x" "," "x" "," "x"))

Which produces this output:

Parse:            ("x" "," "x" "," "x" "," "x") || (BUF)               
Parse:                ("," "x" "," "x" "," "x") || (BUF "x")           
Parse:                    ("x" "," "x" "," "x") || (BUF)               
Parse:                        ("," "x" "," "x") || (BUF "x")           
Parse:                            ("x" "," "x") || (BUF "x")           
Parse:                                ("," "x") || (BUF "x" "x")       
Parse:                                    ("x") || (BUF "x" "x")       
Parse:                                      nil || (BUF "x" "x" "x")

As you can see, the second column was clipped once, but grew every subsequent time.

If you uncomment the setcdr line, this works as expected (output is as follows). You can even dump the setq. I understand why this fixes it, but not why the original error happens in the first place.

Parse:            ("x" "," "x" "," "x" "," "x") || (BUF)               
Parse:                ("," "x" "," "x" "," "x") || (BUF "x")           
Parse:                    ("x" "," "x" "," "x") || (BUF)               
Parse:                        ("," "x" "," "x") || (BUF "x")           
Parse:                            ("x" "," "x") || (BUF)               
Parse:                                ("," "x") || (BUF "x")           
Parse:                                    ("x") || (BUF)               
Parse:                                      nil || (BUF "x")

BTW, the behavior is the same even if i turn off lexical scoping.

Note that `defun` does define GLOBAL functions, not local functions. For local functions one can use `cl-flet` or `cl-labels`. — Rainer Joswig, Jun 12 '16 at 19:33
Possible duplicate of [Unexpected persistence of data](http://stackoverflow.com/questions/18790192/unexpected-persistence-of-data) — Joshua Taylor, Jun 12 '16 at 22:57
And other related Q&As: http://emacs.stackexchange.com/q/20535 http://stackoverflow.com/q/36964728 — phils, Jun 12 '16 at 23:05

score 5 · Accepted Answer · answered Jun 12 '16 at 18:52

5

You cannot mutate literal data, such as '(BUF), and expect sane results. And in your code, the nconc is a mutating operation.

In terms of the "why" you're seeing the behaviour you're seeing, it's due to the (setq buf '(BUF)) expression. That sets it to the same object each time, because it's a literal datum—which you're not supposed to mutate with the likes of nconc. If you changed it to (setq buf (list 'BUF)), then it would generate a new object each time, and you can safely nconc that.

answered Jun 12 '16 at 18:52

C. K. Young

219,335
46
382
435

Would it also work to use a backquote to construct the list, since it's whole purpose in life is to construct new lists? – Joel M Ward Jun 14 '16 at 01:56
@JoelMWard From some brief testing, Emacs seems to follow the same rules as Scheme in this regard, so I'll quote you what R7RS says about it: _A quasiquote expression may return either newly allocated, mutable objects or literal structure for any structure that is constructed at run time during the evaluation of the expression. **Portions that do not need to be rebuilt are always literal.**_ (Emphasis mine.) – C. K. Young Jun 14 '16 at 02:43
@JoelMWard In other words, unless you know how quasiquote expressions are built, it's best to assume it's immutable too. But if you know what you're doing, you can get away with it. :-) In the case of your code above, `\`(,'BUF)` works (from my testing), but not `\`(BUF)`. – C. K. Young Jun 14 '16 at 02:45
It seems like this could be considered sort-of-like-static variables in C. You get one shot to initialize then they're the same from then on. Of course, as we've seen, this is implementation dependent and also probably extremely hard to keep track of as a programmer. This sort of bug could probably be minimized if the literal data was also constant, as in C. At least the error message would point you in the right direction. – Joel M Ward Jun 14 '16 at 20:38
Making all quoted data constant would break all manner of things; so you'd need a different syntax for defining constant data, at which point you need to remember to *use* that syntax, at which point you're back where you started -- needing to know when you can and can't quote data the normal way. – phils Jun 14 '16 at 23:07
Rather than point the finger at the `'(BUF)`, I'd point the finger at `nconc`. Just always use `append` unless you *really* know what you're doing. – Stefan Jun 15 '16 at 21:44

score 0 · Answer 2 · answered Jun 15 '16 at 21:47

This is because of the way data identity works in Elisp. If you replace nconc with append you'll be isolated from those problems. I strongly recommend you stay away from nconc (and friends like setcdr) in general, and especially as long as you're not really familiar with Elisp.

Why is this global variable only able to be set twice, and no more?

2 Answers2