2

Are there guidelines to estimate beforehand which solver should be used for a mathematical optimization project?

Specifically this leads me to the following sub-questions:

  1. How do performance benchmarks translate to probable or even possible optimization-performance-limits - specifically I mean maximum variable numbers for LPs and MILPs? (I am very much aware that this depends on the optimization problem, but there could to be a number(-range) from personal experience. The benchmark I am referring to is by H. Mittelmann: http://plato.asu.edu/ftp/milpc.html.)
  2. Is it a fair conclusion to say CPLEX is 1.42 times slower than GUROBI, and if you are using CPLEX, but not switching to GUROBI, that you are wasting your company's resources (disregarding investments)?
  3. Is it a fair conclusion to state that SAS is an incalculable risk for an optimization-project, because it fails 3 out of 87 calculation-runs when GUROBI doesn't?

I am aware that this question is awkward, but I think it has practical relevance (at least for me). Thanks in advance for answers.

r.user.05apr
  • 5,356
  • 3
  • 22
  • 39
  • 5
    You should benchmark using your own models. They will likely behave very differently than the Mittelmann set. Almost surely your model will not replicate this 1.42 difference ratio. Also there may be good reasons to pick a supplier other than just some raw benchmark speed of the MIP solver. – Erwin Kalvelagen Apr 20 '18 at 15:21
  • 2
    Totally agree with Erwin's comments. If you look at the full set of 'standard' benchmarks, you will find cases where CPLEX beats all the others, and some where Xpress beats the others. So even if there is a net advantage for one solver on average, the variability between problems means that you should try with your specific problem if such small differences matter to you. Further, even within your own problem instances you may find differences between the solvers. We have had cases where e.g. Monday's problem works better on CPLEX, but Thursday's is better with Gurobi for the same client! – TimChippingtonDerrick Apr 21 '18 at 09:11
  • 2
    Further, what matters in business is getting a system that is actually used. So issues like integration into the supporting infrastructure, databases and other systems becomes really important. I have seen customers who have had good 'technical' solutions for a problem, but which never got used in practice because they required extra effort from the users. Your objective should be to deliver a system that is easier to use than the alternatives, so then the users will use it, and the business will see real benefits. The best solver doesn't help you if it doesn't get used in practice. – TimChippingtonDerrick Apr 21 '18 at 09:35
  • 1
    I'd like to add one more point: If you know what kind of problems you need to solve, there is a very high probability that you can get significantly better performance by doing some tuning to find better parameter settings - this is the case for all solvers. – mattmilten Apr 22 '18 at 19:21
  • For a plot of a dozen plain LP (not MIP) solvers, simplex and IP, on 23 Mittelmann benchmarks, see under [my gists](https://gist.github.com/denis-bz). However the above comments are far more important. – denis Jan 24 '21 at 15:24

0 Answers0