12

I'm trying to work through Stuart Halloway's book Programming Clojure. This whole functional stuff is very new to me.

I understand how

(defn fibo[]
    (map first (iterate (fn [[a b]] [b (+ a b)]) [0 1])))

generates the Fibonacci sequence lazily. I do not understand why

(last (take 1000000 (fibo)))

works, while

(nth (fibo) 1000000)

throws an OutOfMemoryError. Could someone please explain how these two expressions differ? Is (nth) somehow holding on to the head of the sequence?

Thanks!

Josh
  • 2,077
  • 1
  • 17
  • 21
  • 1
    Neither of these work for me on tryclj.com as the number is so big it causes an overflow. AFAICT you don't keep a reference to anything so I don't believe anything is "holding onto the head". Are you sure it isn't just because the number is so mindbogglingly big? – Adrian Mouat Dec 15 '11 at 15:57
  • The implementation of last is a straight forward O(n) tail-recursive implementation, and it doesn't hold onto anything. nth is implemented in Java and I'm pretty sure that it doesn't hold onto anything either. Therefore, both of your sequences should work just fine (in theory). The only thing I can think of, though I'm not clear on this affects the outcome, is that your nth call actually calculates 1 item more than your last call. nth 1000000 = 1000001st item (0 indexing) – vedang Dec 15 '11 at 16:42
  • @vedang Thanks... I wouldn't have caught that important distinction. It wasn't the source of my problem, although I hadn't realized that the argument to take is the size of the sequence, while the argument to nth is the index. – Josh Dec 15 '11 at 18:09

3 Answers3

4

I think you are talking about issue that was discussed in google group and Rich Hickey provided patch that solved the problem. And the book, whick was published later, didn't cover this topic.

In clojure 1.3 your nth example works with minor improvements in fibo function. Now, due to changes in 1.3, we should explicitly flag M to use arbitrary precision, or it falls with throwIntOverflow.

(defn fibo[]
  (map first (iterate (fn [[a b]] [b (+ a b)]) [0M 1M])))

And with these changes

(nth (fibo) 1000000)

succeed (if you have enough memory)

4e6
  • 10,696
  • 4
  • 52
  • 62
  • I was using a snapshot version distributed with the book, 1.1.0-alpha-SNAPSHOT. Changing to 1.3.0 worked. I'm guessing the version I had contained the bug that you refer to... namely that "A recent optimization attempt introduced a head-retaining hop in RT.nth" – Josh Dec 15 '11 at 18:05
  • The "M" though wasn't required. Clojure seems to be converting to BigInt when needed, at least with 1.3.0. – Josh Dec 15 '11 at 18:11
  • @Josh, not on my machine. See my answer. – Michiel Borkent Dec 15 '11 at 18:22
1

What Clojure version are you using? Try (clojure-version) on a repl. I get identical results for both expressions in 1.3.0, namely an integer overflow.

For

(defn fibo[]
    (map first (iterate (fn [[a b]] [b (+ a b)]) [(bigint 0) 1])))

I get correct results for both expressions (a really big integer...).

Michiel Borkent
  • 34,228
  • 15
  • 86
  • 149
0

I think that you may be hitting a specific memory limit for your machine, and not a real difference in function.

Looking at the source code for nth in https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java it does not look like either nth or take are retaining the head.

However, nth uses zero-based indexing, rather than a count by item number. Your code with nth selects the 1000001st element of the sequence (the one at index 1000000). You code with take is returning the final element in a 1000000 element sequence. That's the item with the index 999999. Given how fast fib grows, that last item could be the one that broke the camel's back.

Also, I was checking the 1.3.0 source. Perhaps earlier versions had different implementations. To get your fibo to work properly in 1.3.0 you need to use the arithmetic functions that will promote numbers to bignums:

(defn fibo[]
    (map first (iterate (fn [[a b]] [b (+' a b)]) [0 1])))
modified.dog
  • 46
  • 5