0

I've recently begun to use the scipy.optimize minimisation library, and am experimenting with different algorithms.

I see there's good documentation for each algo individually on the docs page, but does anybody know of any intercomparison work that's been done (particularly with reference to those available to scipy)?

For instance, I'm finding that SLSQP is much faster than L-BFGS-B but often doesn't do quite as well..

Robbie Mallett
  • 131
  • 1
  • 11
  • Depends heavily on use-cases. I could have sworn, that L-BFGS-B is dominating SLSQP completely in most cases in regards to speed (only), which would also be expected theory-wise (for starters: it's less general; unconstrained). I would be careful with ` SLSQP is much faster` without saying what that means (e.g. *local-optimum* with `1e-8 objective tolerance`) and `often doesn't do quite as well..` does not make much sense as both are local-optimizers and should convergence to the same optimum. If that's not the case, there is probably a problem in the model (e.g. not twice diff; not scaled). – sascha Dec 14 '20 at 10:23
  • Hi Sascha, yeah it's possible that there is a problem in the model then! Could you drop a link or briefly explain what you mean by 'twice diff' and 'scaled'? – Robbie Mallett Dec 15 '20 at 11:30
  • Most optimizers (within scipy.optimize) assume that the objective (and constraints if part) are twice [differentiable](https://en.wikipedia.org/wiki/Differentiable_function), a common thing in general nonlinear-optimization. This renders lots of stuff wrong as used by people (`if` in functions; `abs` and so on). *Scaling* is somewhat the process of *balancing* model and data such it's *nice* in regards to numerics. Sometimes you calculate with seconds and the algebra needs to compare values like `1e8, 1e1` (bad due to fp-calc) in gradient-calc which would be different if *days* would be used. – sascha Dec 15 '20 at 11:35

0 Answers0