lazy-sequence with recursion: stack-overflow on certain cases with repl

Question

I play with clojure, more precisely with lazyness. I tried this chunk of code:

(defn even-numbers
  ([] (even-numbers 0))
  ([n] (cons n (lazy-seq (even-numbers (+ n 2))))))
(take 1 (even-numbers 0))

It's quite idiomatic, I don't think there is something wrong with this.

The problem is I use cider within emacs. If I first evaluate the defn and afterwards evaluate the function call I get a stack overflow. Evaluating the whole buffer is OK.

I suspect that enlighten mode is to be blamed since I don't get the overflow when I disable it. I just want to understand what happens here.

[EDIT] I'm quite sure the issue is related to the enlighten mode of cider. disabling it I never get a stack overflow.

To dig a little bit into it, considering the slightly modified excerpt:

(ns clojure-noob.bizarerrie)

(defn even-numbers
  ([] (even-numbers 0))
  ([n] (do (println (even-numbers (+ n 2)))
           (cons n (lazy-seq (even-numbers (+ n 2)))))))
(take 1 (even-numbers 0))

which also gives a stack overflow in every cases. Since enlighten mode try to resolves expression to output it during execution on the fly; Intuitively I'd say it tries to print the (even-numbers (+ n 2)) form (like the println I just added), which cause the stack overflow.

score 1 · Answer 1 · answered Sep 11 '18 at 18:03

you have the cons and lazy-seq switched in the lazy sequence definiton:

Here it is with the cons inside the lazy-seq

user> (defn even-numbers [n]
        (do
          (lazy-seq (cons (do
                            (printf "creating value %s for lazy sequence" n)
                            n)
                          (even-numbers (+ n 2))))))
#'user/even-numbers

And again with the cons outside the lazy-seqwhich creates an almost lazy sequence

user> (defn even-numbers-not-really-lazy [n]
        (do
          (cons (do
                  (printf "creating value %s for lazy sequence" n)
                  n)
                (lazy-seq (even-numbers (+ n 2))))))

A proper lazy sequence should to no work when it's created, let's test both of these:

#'user/even-numbers-not-really-lazy
user> (def lazy-even-numbers
        (even-numbers 0))
#'user/lazy-even-numbers

Yep, that one looks good, now with the cons on the outside:

user> (def not-lazy-even-numbers
        (even-numbers-not-really-lazy 0))
creating value 0 for lazy sequence
#'user/not-lazy-even-numbers

hmmm looks like we did some work when we created the cons cell.

That is a good point. If element of lazy sequence is a result of some expensive calculation, the cons should be inside lazy-seq. Otherwise it's not a big deal. — Simon Polak, Sep 13 '18 at 01:53

Simon Polak · Answer 2 · 2019-06-25T00:02:22.640

0

You ~~get a stack overflow~~ are running out of heap space, because you are trying to realize the infinite lazy sequence. You need to specify how many values you want realized in that sequence by calling function called take

(take 10 (even-numbers 0)) ; if you want to realize first 10 values

edited Jun 25 '19 at 00:02

answered Sep 11 '18 at 05:17

Simon Polak

1,959
1
13
21

2

Realizing the entire lazy sequence wouldn't cause a stack overflow, though. Maybe printing it would, although to be honest I don't see why. What *should* happen is that Clojure prints this thing forever until your REPL runs out of memory. – amalloy Sep 11 '18 at 05:40
applying take doesn't change anything. Actually that was what I wrote originally but I removed it during my tinkering. I will edit the original excerpt to avoid confusion on this point. – Momh Sep 11 '18 at 05:50
If you don't believe me. Try evaluating this in your repl: (range) – Simon Polak Sep 11 '18 at 06:10
@blushrt What repl, though? Try it with a bare clojure.jar and that will execute until you run out of *heap*, not stack. You can prevent running out of heap too if you are careful not to tie the result to the special repl var `*1`, e.g. by evaluating `(print (range))` instead - this should be able to run until it gets to Long/MAXIMUM_VALUE or you get bored. If your repl runs out of stack on this it's because you're using some broken repl tooling. – amalloy Sep 11 '18 at 06:18
Actually you are right, it wouldn't cause stack overflow, it would cause OutOfMemmory error, because you would run out of JVM heap space. You could print the infinite sequence only by discarding previous values by using dorun function. – Simon Polak Sep 11 '18 at 06:18
@Momhain, you can prevent running out of heap only by consuming the infinite sequence one element at a time (while not retaining the head of the sequence). See how print is implemented, it calls pr under the hood, which recurisivelly calls first on the sequence. No it's not about tooling, it's about understanding how sequences work, I suggest you read the Joy of Clojure book, it has a great section on lazy seqs. – Simon Polak Sep 11 '18 at 06:31
Your comments are very interesting although abit too technical for me a.t.m. I get the idea of eating all the heap with such constructs, although my issue is different since I observe a stack overflow and not a OutOfMemory. Intuitively I'd say that enlighten mode from the cider plugin tries to consumes the whole sequence (see edit) – Momh Sep 11 '18 at 06:48

lazy-sequence with recursion: stack-overflow on certain cases with repl

2 Answers2