reduce and map on accumulator produces stack overflow

Question

Why do I need to replace map with mapvin this piece of code to prevent a stack overflow:

#!/bin/bash lein-exec                                                           


(println (reduce (fn [acc _]                                                    
                   ;;(mapv #(inc %) acc))                                       
                   (map #(inc %) acc))                                          
                 (repeat 2 0)                                                   
                 (range (long 1e6))))

~

I don't understand how the acc is processed when lazy. Thanks for insight.

OlegTheCat · Accepted Answer · 2016-09-22T11:03:49.933

Basically, what you are getting is a large number of nested lazy sequences, which, when poked, cause a stack overflow.

Let's look at smaller example:

(reduce (fn [acc _]
          (map inc acc))
        (repeat 2 0)
        (range 3))

Since map is lazy, the result of above will be the next:

(map inc (map inc (map inc (0 0)))

So you are not eagerly mapping acc with inc, but only putting lazy sequences one into another, which will be realized afterwards.

Going back to the original example where range takes 1e6, the result will be the following:

(map inc 
    (map inc 
         (<... rougly 1e6 nested lazy sequences here ...>
            (map inc (0 0))) ...)

Realizing of this will consume about 1e6 stack frames, which will definitely cause stack overflow.

In case of mapv laziness is not involved and acc is realized right away, so the result of your example will be [1000000 1000000] right after reduce finishes.

reduce and map on accumulator produces stack overflow

1 Answers1