-1

I wrote the code below for gradient descent algorithm. I get an error, can anyone tell me why and how I can fix it?

gradient <- function(h, start, alpha = 0.01, tolerance = 0.0001, debug = FALSE) {      
    MAXITER <- 1000
    x_old <- start      
    iter <- 0    

    cat("gradient descent minimization =\n")
    if (debug == TRUE) cat("iter=", iter,", value=", x_old, "\n")

    repeat {
        iter <- iter + 1
        x_new <- x_old - alpha*h(x_old)
        if (debug == TRUE) cat("iter=", iter,", value=", x_new, "\n")
        if (abs(x_old - x_new) < tolerance) break    
        if (iter > MAXITER) break
        x_old <- x_new    
    }
    cat("total number of interations =", iter, "\n")
    cat("last diference =", abs(x_old - x_new), "\n")
    cat("final value =", x_new, "\n")
    cat("final function value =", h(x_new), "\n")    
}

h <- function(x){ x^4 - 8*x^2 + 2*x }

gradient(h, -5, tolerance = 0.0001)

...

gradient descent minimization =

Error in if (abs(x_old - x_new) < tolerance) break :     
  missing value where TRUE/FALSE needed
Zheyuan Li
  • 71,365
  • 17
  • 180
  • 248
Rilja
  • 37
  • 5
  • Is it so hard to select your code and press Ctrl-K to indent? – smci Nov 01 '16 at 23:34
  • 1
    Stylistic point: instead of `iter <- 0; ... repeat { iter <- iter + 1; ... if (iter > MAXITER) break ...}` is just a for-loop in disguise: `for (iter in 1:MAXITER) { ... }` – smci Nov 01 '16 at 23:43
  • Why didn't you try calling `gradient(..., debug = TRUE)`? The print statements make the answer jump out immediately. Is this really your own code? I doubt it. – smci Nov 01 '16 at 23:51
  • 1
    You can visualize your [gradient function h in WolframAlpha](https://www.wolframalpha.com/input/?i=x%5E4+-+8*x%5E2+%2B+2*x) , and this will help you debug what should happen if you start at x_start = -5. Actually, we should look at [the integral of h dx](https://www.wolframalpha.com/input/?i=integ+(x%5E4+-+8*x%5E2+%2B+2*x)+dx), not the gradient h. – smci Nov 01 '16 at 23:57

1 Answers1

0

I just run your code. There is a numeric instability here. Add a condition if(is.nan(x_new)) break after your x_new <- x_old - alpha*h(x_old) statement.

Jim Raynor
  • 198
  • 2
  • 9
  • I think it works but I get a different error now: > gradient(h, -5, tolerance = 0.0001) gradient descent minimization = total number of interations = 7 Error in abs(x_old - x_new, "\n") : 2 arguments passed to 'abs' which requires 1 Called from: cat("last diference =", abs(x_old - x_new, "\n")) Browse[1]> Q – Rilja Nov 01 '16 at 23:57
  • There is a syntax error in `cat("last diference =", abs(x_old - x_new, "\n"))` it should be `cat("last diference =", abs(x_old - x_new), "\n")` – Jim Raynor Nov 02 '16 at 00:05
  • Actually with the way the code is set up you need to check two steps up front. So replace `if(is.nan(x_new)) break` with `if(is.nan(h(x_new - alpha*h(x_new)))) break`. – Jim Raynor Nov 02 '16 at 00:07