SICP 1.45 - Why are these two higher order functions not equivalent?

Question

I'm going through the exercises in [SICP][1] and am wondering if someone can explain the difference between these two seemingly equivalent functions that are giving different results! Is this because of rounding?? I'm thinking the order of functions shouldn't matter here but somehow it does? Can someone explain what's going on here and why it's different?

Details:

Exercise 1.45: ..saw that finding a fixed point of y => x/y does not converge, and that this can be fixed by average damping. The same method works for finding cube roots as fixed points of the average-damped y => x/y^2. Unfortunately, the process does not work for fourth roots—a single average damp is not enough to make a fixed-point search for y => x/y^3 converge.

On the other hand, if we average damp twice (i.e., use the average damp of the average damp of y => x/y^3) the fixed-point search does converge. Do some experiments to determine how many average damps are required to compute nth roots as a fixed-point search based upon repeated average damping of y => x/y^(n-1).

Use this to implement a simple procedure for computing the roots using fixed-point, average-damp, and the repeated procedure of Exercise 1.43. Assume that any arithmetic operations you need are available as primitives.

My answer (note order of repeat and average-damping):

(define (nth-root-me x n num-repetitions)
  (fixed-point (repeat (average-damping (lambda (y)
                                           (/ x (expt y (- n 1)))))
                       num-repetitions)
               1.0))

I see an alternate web solution where repeat is called directly on average damp and then that function is called with the argument

(define (nth-root-web-solution x n num-repetitions)
      (fixed-point
         ((repeat average-damping num-repetition)
          (lambda (y) (/ x (expt y (- n 1)))))
         1.0))

Now calling both of these, there seems to be a difference in the answers and I can't understand why! My understanding is the order of the functions shouldn't affect the output (they're associative right?), but clearly it is!

> (nth-root-me 10000 4 2)
> 
> 10.050110705350287
> 
> (nth-root-web-solution 10000 4 2)
> 
> 10.0

I did more tests and it's always like this, my answer is close, but the other answer is almost always closer! Can someone explain what's going on? Why aren't these equivalent? My guess is the order of calling these functions is messing with it but they seem associative to me.

For example:

(repeat (average-damping (lambda (y) (/ x (expt y (- n 1)))))
         num-repetitions)

vs

((repeat average-damping num-repetition)
 (lambda (y) (/ x (expt y (- n 1)))))

Other Helper functions:

(define (fixed-point f first-guess)
  (define (close-enough? v1 v2)
    (< (abs (- v1 v2))
       tolerance))
  (let ((next-guess (f first-guess)))
    (if (close-enough? next-guess first-guess)
        next-guess
        (fixed-point f next-guess))))

(define (average-damping f) 
  (lambda (x) (average x (f x))))

(define (repeat f k)
  (define (repeat-helper f k acc)
    (if (<= k 1)
        acc
           ;; compose the original function with the modified one
        (repeat-helper f (- k 1) (compose f acc)))) 
  (repeat-helper f k f))

(define (compose f g)
  (lambda (x)
    (f (g x))))

Answer 1

You are asking why “two seemingly equivalent functions” produce a different result, but the two functions are in effect very different.

Let’s try to simplify the problem to see why they are different. The only difference between the two functions are the two expressions:

(repeat (average-damping (lambda (y) (/ x (expt y (- n 1)))))
        num-repetitions)

((repeat average-damping num-repetition)
   (lambda (y) (/ x (expt y (- n 1)))))

In order to simplify our discussion, we assume num-repetition equal to 2, and a simpler function then that lambda, for instance the following function:

(define (succ x) (+ x 1))

So the two different parts are now:

(repeat (average-damping succ) 2)

and

((repeat average-damping 2) succ)

Now, for the first expression, (average-damping succ) returns a numeric function that calculates the average between a parameter and its successor:

(define h (average-damping succ))
(h 3) ; => (3 + succ(3))/2 = (3 + 4)/2 = 3.5

So, the expression (repeat (average-damping succ) 2) is equivalent to:

(lambda (x) ((compose h h) x)

which is equivalent to:

(lambda (x) (h (h x))

Again, this is a numeric function and if we apply this function to 3, we have:

((lambda (x) (h (h x)) 3) ; => (h 3.5) => (3.5 + 4.5)/2 = 4

In the second case, instead, we have (repeat average-damping 2) that produces a completely different function:

(lambda (x) ((compose average-damping average-damping) x)

which is equivalent to:

(lambda (x) (average-damping (average-damping x)))

You can see that the result this time is a high-level function, not an integer one, that takes a function x and applies two times the average-damping function to it. Let’s verify this by applying this function to succ and then applying the result to the number 3:

(define g ((lambda (x) (average-damping (average-damping x))) succ))
(g 3) ; => 3.25

The difference in the result is not due to numeric approximation, but to a different computation: first (average-damping succ) returns the function h, which computes the average between the parameter and its successor; then (average-damping h) returns a new function that computes the average between the parameter and the result of the function h. Such a function, if passed a number like 3, first calculates the average between 3 and 4, which is 3.5, then calculates the average between 3 (again the parameter), and 3.5 (the previous result), producing 3.25.

Answer 2

The definition of repeat entails

  ((repeat f k) x)  =  (f (f (f (... (f x) ...)))) 
                    ;   1  2  3       k

with k nested calls to f in total. Let's write this as

                    =  ((f^k) x)

and also define

  (define (foo n)  (lambda (y)  (/ x (expt y (- n 1)))))
  ;       ((foo n)  y)  =  (/ x (expt y (- n 1)))

Then we have

  (nth-root-you x n k)  =  (fixed-point  ((average-damping (foo n))^k)   1.0)

  (nth-root-web x n k)  =  (fixed-point  ((average-damping^k)  (foo n))  1.0)

So your version makes k steps with the once-average-damped (foo n) function on each iteration step performed by fixed-point; the web's uses the k-times-average-damped (foo n) as its iteration step. Notice that no matter how many times it is used, a once-average-damped function is still average-damped only once, and using it several times is probably only going to exacerbate a problem, not solve it.

For k == 1 the two resulting iteration step functions are of course equivalent.

In your case k == 2, and so

(your-step y)  =  ((average-damping (foo n)) 
                     ((average-damping (foo n))  y))                 ; and,

(web-step  y)  =  ((average-damping (average-damping (foo n)))  y)

Since

((average-damping f)  y)  =  (average  y  (f y))

we have

(your-step y)  =  ((average-damping (foo n)) 
                     (average  y  ((foo n)  y)))
               =  (let ((z (average  y  ((foo n)  y))))
                     (average  z  ((foo n)  z)))

(web-step  y)  =  (average y  ((average-damping (foo n))  y))
               =  (average y   (average y   ((foo n)  y)))
               =  (+ (* 0.5 y)  (* 0.5 (average y  ((foo n)  y))))
               =  (+ (* 0.75 y)  (* 0.25 ((foo n)  y)))
    ;; and in general:
    ;;         =    (2^k-1)/2^k * y  +  1/2^k * ((foo n)  y)

The difference is clear. Average damping is used to dampen the possibly erratic jumps of (foo n) at certain ys, and the higher the k the stronger the damping effect, as is clearly seen from the last formula.