0

At the moment I'm trying to do a minimization (optimization) problem in R, where I have a vector X1 that I want to approximate through a weighted average of a matrix X2 and a vector of weights w. That means I want to minimize 

wg <- function(w)
{
    t(X1 - X2 %*% w) %*% (X1 - X2 %*% w)
}

the constraints on the weights are w[i]>= 0 and sum(w) = 1 .

At the moment I'm using the DEoptim package to do my optimization, but I feel like it can't deal well with corner solutions.

I'm replicating a method that was used in an economics paper and in that paper almost all of the weights turned out to be zero. I expected a similar result in my case ( I want to model Arizona through a weighted average of the other states), especially due to the heterogeneity in the economic situation.

At the moment I feel like it's more of a problem with the DEoptim package than with my methodology and I don't really trust the results. Which other package can I use, preferrably ones that are stronger in looking for corner solutions?

my DEoptim is set up as follows:

controlDE <- list(reltol=.0000000001,steptol=150, itermax = 5000,trace = 250)
#control parameters

outDEoptim <- DEoptim(fn = wg, lower = rep(0, N), upper = rep(1, N), 
    control = controlDE)

Any help would be much appreciated!

Julius
  • 15
  • 5
  • Can't this simply be cast as a nonlinear least squares problem, by using a transformation on w to guarantee non-negativity? – Glen_b Feb 14 '13 at 00:11
  • the non-negativity is not a problem, since I set the lower bound to 0 in DEoptim. However I feel like this lower bound is exactly why a solution is not forthcoming in the DEoptim package. unfortunately I'm not that good yet in understanding how exactly the different computational solvers work and so I can't pick the one which would be best for this kind of problem. – Julius Feb 14 '13 at 00:18
  • Alright, I just did a super ghetto solution since a deadline is looming and I have to get on with it, but reducing the number of parameters through iteration by hand is actually producing sensible results. I picked the 9 highest rated factors and dropped everything else. then I redid my weights and the minimized value was far lower, confirming my suspicion that the algorithm is not good with corner solutions ( leaving a factor out is the same as forcing it to zero in this context). – Julius Feb 14 '13 at 10:57

1 Answers1

1

A stochastic solver such as DEoptim will by nature have difficulties finding optimal solutions on lower dimensional subsets such as the one defined by sum(w) = 1.

There is a first, not quite correct way of doing this by reducing the problem to (n-1) dimensions by setting w <- c(w, 1-sum(w)). The last component might get less than 0, but normally it won't. Now apply DEoptim or optim:

set.seed(1357); m <- 4; n <- 5
X2 <- round(matrix(runif(20), m, n), 2)
X1 <- X2 %*% c(0, 1, 0, 0, 0)               # solution c(0,1,0,0,0)

wg <- function(w) {                         # length(w) == 4
    w <- c(w, 1 - sum(w))
    t(X1 - X2 %*% w) %*% (X1 - X2 %*% w)    # sum((X1 - X2 %*% w)^2)
}

w0 <- rep(1/n, n-1)                         # initial point (1/n, ..., 1/n)
optim(w0, wg, lower = rep(0, n), upper = rep(1, n),
        method = "L-BFGS-B", control = list(factr = 1e-8))
## $par
## [1] 0 1 0 0                              # wmin = c(0,1,0,0,0)

Or you apply one of the solvers in R that can handle equality constraints, for example Rdonlp2 (on R-Forge), auglag in package alabama, or slsqp in package nloptr. But I feel this would be overshooting.

Hans W.
  • 1,799
  • 9
  • 16
  • thanks for the help. so far I implemented the equality constraint through a penalty term at the end. after reducing the number of factors by hand that actually guaranteed a sum of 1. it turns out that I could force all but 4 of the factors to 0 by omitting them in the formula and only afterwards did I lose accuracy. as I said above, I'm on a deadline, so I have to take what I can get and just get a reasonable workable result :) Thank you anyways, I will look at those otehr packages! – Julius Feb 14 '13 at 11:14
  • I just used the optim package now and to much better results! – Julius Feb 15 '13 at 17:59