0

Currently, I'm using a primal-dual method to minimize a quadratic problem with simple linear constraints (specifically, x >= 0). For the termination condition, I'm currently using the standard: ie error "e" term must be less than a threshold value.

However, to compute the "e" term is relatively expensive, as I need to evaluate the full quadratic function. Is it possible to have a simpler/faster termination condition, such as one the following:

1) if the sum of all the changes in x is below a certain threshold, then terminate; or, 2) if the max change in x is below a certain threshold, then terminate?

Intuitively, that makes sense to me, but I've found that my intuition is wrong half the time with these minimization problems, so I'm not sure if my intuition is theoretically sound or not....

Jonathan Sylvester
  • 1,275
  • 10
  • 23

1 Answers1

1

Practice 1:

For the termination condition, I'm currently using the standard: ie error "e" term must be less than a threshold value. Aren't you assuming, that the objective can be pushed to 0 here? How do you determine this threshold a-priori?

In regards to theory:

  • 1) if the sum of all the changes in x is below a certain threshold, then terminate
    • this is a heuristic (might be a good idea to take an outer absolute-value) and i think i saw something like that in other codes)
    • it has no theoretical guarantees; not sufficient, but a necessary condition
  • 2) if the max change in x is below a certain threshold, then terminate
    • an often used heuristic
    • again: no theoretic guarantees; not sufficient, but a necessary condition

  • You should look into the Karush–Kuhn–Tucker conditions (first-order necessary conditions for a solution)

    • The kind of KKT-conditions you would need depend on your problem: convex vs non-convex -> Q psd or not (edit it seems yours is convex according to your tags!)
    • Interesting read with special-treatment of your problem

    • Remark: your problem is a box-constrained QP which is actually simpler than inequality-constrained QP! This is important!

      • Why important?: because projected unconstrained optimization approaches will work too and unconstrained optimization is much easier to do. Additionaly, important in this case: your projection seems to be cheap compared to calculating the objective (projection is linear in your case)!
      • Check out pages 8- 11

Now this might effect in some heavy theory-work followed by some implementation. Consider using tools already available!

Practice 2:

Have a look at box-constrained QP-solvers (convex or non-convex; depending on your problem). These should be available and will save you trouble!

You could even use the nearly everywhere available LBFGS-B which might already hard to beat, if you give it gradients (opposed to numerical-differentiation). Although it's much more general, it's often very powerful! (in python, using scipy, using this would be a few lines)

L-BFGS-B is a limited-memory quasi-Newton code for bound-constrained optimization, i.e. for problems where the only constraints are of the form l<= x <= u.

There are probably other already available alternatives too. Trust Region Reflective and co.

(Side-note: i wonder how the evaluation-cost can be that costly, as you describe, compared to the other operations in your primal-dual (IPM?) algorithm)

sascha
  • 32,238
  • 6
  • 68
  • 110