QuadraticMinimizer always returns a non-sensical answer

Question

I'm having trouble with the QuadraticMinimizer- every time I execute it I get errors that are non-sensical. I've looked at the scaladoc and the code and I am just not sure what I am doing incorrectly.

I have the following code:

    val lBounds:DenseVector[Double] = DenseVector(Array(Double.NegativeInfinity, 5.0, 0.1, 50.0, 50.0))
  val uBounds:DenseVector[Double] = DenseVector(Array(Double.PositiveInfinity, 500.0, 100.0, 99000.0, 99000.0))

  val inputMatrix:DenseMatrix[Double] = breeze.linalg.csvread(file=new java.io.File(getClass.getResource("/input/zip06latlong.csv").toURI), skipLines=1)

  val y = inputMatrix(::, 0).toDenseVector
  val X = inputMatrix(::, Seq(1,2,3,4,5)).copy.toDenseMatrix
  val T = X.toDenseMatrix.t

  val gram = (T * X)
  val b:Transpose[DenseVector[Double]] = y.t * X

  println(gram)
  println(b.inner)
  breeze.linalg.csvwrite(new java.io.File("grammatrix.csv"), gram)
  val minimizer:QuadraticMinimizer = new QuadraticMinimizer(rank(gram), ProjectBox(lBounds,uBounds))


  val coeffs =  minimizer.minimize(gram, b.inner)
  println(coeffs)

here's what the gram looks like:

    279.0      628207.0       1461245.0        1024.0     729.5      
628207.0   1.569309427E9  3.414471724E9    2449533.0  1755536.5  
1461245.0  3.414471724E9  1.2324155401E10  5511816.0  3846583.0  
1024.0     2449533.0      5511816.0        3980.0     2786.0     
729.5      1755536.5      3846583.0        2786.0     2092.75

and the b vector is:

2.917264069193999E8, 7.294450468242601E11, 1.585917338779061E12, 1.131888709844E9, 8.260072757806E8

when I execute the code I get:

DenseVector(-12171.118011368422, -424.79971124882286, -9.565028484748783, 49.3827769217138, 49.536905925364195)

Which is below the lower bounds specified.

score 0 · Accepted Answer · answered Aug 02 '15 at 19:05

0

Could you please add the H (gram matrix) and y (linear term) as well so that I can test it out ? Also the initialize pattern is for advanced users (like Spark ALS for which I wrote the solver) but you can use the simple pattern as follows (I used your bounds):

val lb = DenseVector(Array(Double.NegativeInfinity, 5.0, 0.1, 50.0, 50.0))
val ub = DenseVector(Array(Double.PositiveInfinity, 500.0, 100.0, 99000.0, 99000.0))

val n = 5

val ata = new DenseMatrix[Double](5, 5,
  Array(4.377, -3.531, -1.306, -0.139, 3.418,
    -3.531, 4.344, 0.934, 0.305, -2.140,
    -1.306, 0.934, 2.644, -0.203, -0.170,
    -0.139, 0.305, -0.203, 5.883, 1.428,
    3.418, -2.140, -0.170, 1.428, 4.684))

val atb = DenseVector(-1.632, 2.115, 1.094, -1.025, -0.636)

val qpSolverBounds = new QuadraticMinimizer(n, ProjectBox(lb, ub))
val result = qpSolverBounds.minimize(ata, atb)
println(s"Bounds test $result")

I am getting result as DenseVector(-33.02749751949581, 5.0, 0.1, 50.0, 50.0) which does not look non-sensical.

Please don't change the default maxIter since this control is given for flows where absolute convergence is not necessary. Let the solver choose the maxIter.

answered Aug 02 '15 at 19:05

Debasish Das

50
1
5

The gram matrix I'm using can be computed by taking columns 1:4 of the input and multiplying them by their transpose, I'll add the original to the question, though it's several hundred rows for the 4 vectors – Steven Fines Aug 03 '15 at 18:00
Please add only the gram matrix and the linear term..if you have 5 variables, gram will be 5 x 5 matrix (not that big)..Also the linear term should have 5 terms in it as well...Those are the H and y that should be send to minimize – Debasish Das Aug 03 '15 at 20:45
Your comment tells me what the problem is, I think. My gram matrix is like 279x279, so apparently I'm constructing it incorrectly. – Steven Fines Aug 03 '15 at 21:04
Here's the gram matrix... (it has the same problem) 3.497096141589097, 7.294450468242473E11,1.5859173387738567,1.13188870983905,8.260072757785255 7.294450468242473,1.569309427E9,3.414471724E9,2449533.0,1755536.5 1.5859173387738567,3.414471724E9,1.2324155401E10,5511816.0,3846583.0 1.13188870983905E9,2449533.0,5511816.0,3980.0,2786.0 8.260072757785255E8,1755536.5,3846583.0,2786.0,2092.75 – Steven Fines Aug 03 '15 at 22:43
I am not sure how many variables your QP has...if it has 5 variables H will be 5x5, y, lb, ub will be vector of length 5...if the problem has 279 variables, H will be 279x279 and y, lb, ub will be vector of length 279...The problem you want to solve is 0.5x'Hx + y'x s.t lb <= x <= ub...Looks the message you are getting is not that descriptive and it is not guiding you towards the right issue...I would like to put a descriptive error message once we figure out the solve issue for your problem... – Debasish Das Aug 04 '15 at 15:50
well, I think that may be the nub of the problem right there... I'm not quite sure what to do to reduce my 279 element vector for y down to 5 elements? Should it be the cos of x*y for each vector? – Steven Fines Aug 04 '15 at 18:12
gram: 279.0 628207.0 1461245.0 1024.0 729.5 628207.0 1.569309427E9 3.414471724E9 2449533.0 1755536.5 1461245.0 3.414471724E9 1.2324155401E10 5511816.0 3846583.0 1024.0 2449533.0 5511816.0 3980.0 2786.0 729.5 1755536.5 3846583.0 2786.0 2092.75 and the following b: 2.917264069193999E8, 7.294450468242601E11, 1.585917338779061E12, 1.131888709844E9, 8.260072757806E8) I get -12171.118011368422, -424.79971124882286, -9.565028484748783, 49.3827769217138, 49.536905925364195) which are below the lower bounds – Steven Fines Aug 04 '15 at 18:26
The gram matrix that you provided has a high condition number 1.0751e+09...QuadraticMinimizer uses a proximal algorithm and such high condition number will affect convergence. Can you do a feature scaling to improve the condition number of the gram matrix by running a zscore normalization on your features ? – Debasish Das Aug 05 '15 at 01:19
The following brings me close to optimal but you can see I have to lower the tolerances and increase the maxIter. Feature scaling will help you. The problem is heavily ill conditioned. val qpSolverBounds = new QuadraticMinimizer(n, ProjectBox(lb, ub), maxIters=400000, abstol = 1e-8, reltol=1e-8) val result = qpSolverBounds.minimizeAndReturnState(H, b)println(s"Bounded Quadratic minimization iters ${result.iter} converged ${result.converged} output ${result.z}") – Debasish Das Aug 05 '15 at 01:50
iters 246692 converged true output DenseVector(-1057654.0153513495, 5.0, 0.1, 50.0, 50.0) The iteration count of 246K is too high...Mosek solution is (-1057710.543614, 5.000000, 0.100000, 50.000051, 50.000054) – Debasish Das Aug 05 '15 at 01:57

QuadraticMinimizer always returns a non-sensical answer

1 Answers1