Is there any way to extract parameters and objective function for each iteration in R optimx

Question

Suppose I have an optimization problem to solve using R, optimx. Is there any way I can extract the parameters and objective function values over time?

f<-function(x){
  return(sum(abs(x)))
}

gr<-function(x){
  return(sign(x))
}
opt=optimx::optimx(runif(2),f,gr,method="BFGS")

The goal is trying to make such plot:

I think we can manually do it with Gradient Decent with following code, but how I can I do it in optimx?

x=c(0.5,1.5)
alpha=0.1
max_iter=20
x_trace=matrix(rep(0,max_iter*2),ncol=2)

for (i in 1:max_iter){
  x=x-alpha*gr(x)
  x_trace[i,]=x
}
f_trace=apply(x_trace,1,f)

Roland · Accepted Answer · 2016-08-22T07:54:45.653

Create a side effect:

f<-function(x){
  .GlobalEnv$i <- get("i", envir = .GlobalEnv) + 1
  .GlobalEnv$log[get("i", envir = .GlobalEnv),] <- x
  return(sum(abs(x)))
}

gr<-function(x){
  return(sign(x))
}

library(optimx)
i <- 0
log <- matrix(numeric(100 * 2), ncol = 2)

opt <- optimx(c(0.8, -0.9),f,gr,method="BFGS")
log <- log[seq_len(i), ]

plot(log, type = "l", xlim = c(-2, 2), ylim = c(-1.2, 1.2))

Note that this includes all function calls, even those where the algorithm rejects the result and retries. control = list(trace = TRUE, REPORT = 1) lets optimx print the function values for accepted tries and you could capture.output this and use it to get only the parameters of these from log.

It would be better to change optimx to return all accepted attempts, but I'm not going to invest that kind of effort. You could ask Prof. Nash if he would be willing to do this, but if you don't have a compelling common use case, he probably is not going to either.

Interesting plot and really helpful for me. one problem with this approach, we do not know how to assign `log` variable, because number of iterations used by toolbox is not certain right? — hxd1011, Aug 22 '16 at 13:30
We can make a reasonable estimate. If the estimate was too small, retry with a bigger estimate. The number of iteration is limited to 100, so something like n = 1000 should be sufficient even for most problematic fits. — Roland, Aug 22 '16 at 13:53

score 2 · Answer 2 · edited May 23 '17 at 12:02

As far as I know, method="L-BFGS-B" can make a progress report of params. But the result doesn't have the report, so I kept the message and extracted the value.

library(optimx); library(dplyr)

cap <- capture.output(optimx(runif(2), f, gr, method="L-BFGS-B", 
                             control=list(trace=6, REPORT=1)))
temp <- cap[grep("X =|X0 =", cap)]
d <- gsub("X0 = |X = |Cauchy X =  ", "", temp) %>% strsplit(" ") %>% 
  unlist() %>% as.numeric() %>% matrix(ncol=2, byrow=T)

plot(-2:2,-2:2, type="n", ann=F)
for(i in c(1,2,4)) polygon(c(-0.5,0,0.5,0, -0.5)*i, c(0, 0.5, 0, -0.5, 0)*i)
points(d, pch=letters[1:nrow(d)])

[edited]
As help says, source code (opt/lbfgs_bcm.shar) helps to exactly understand these values (@Roland commented, thanks !!). And using this approach with method="L-BFGS-B", you can get additional information about what values control=list(trace=6, REPORT=1) reports.

But of course this is a different optimizer with different intermediate results. — Roland, Aug 22 '16 at 10:20

Is there any way to extract parameters and objective function for each iteration in R optimx

2 Answers2