Docplex, large number of iterations ( ItCnt = 1.25e+08)

Question

When running the cplex model "docplex(python)", I noticed that the CPLEX repeats the same gap value several times (more than 300 times). without any improvement and with a large number of nodes left to explore (even if the problem is not big enough, number of nodes : 50, number of variables: 8655, number of constraints: 16950). I m asking if it's a normal results and if there are parameters that can help the solver direct the search, like changing the search space without stopping the solver? (Attached, you can see the log output) [1]: https://www.mediafire.com/file/s9sofxhze8em31y/log_output_2.pdf/file

Answer 1

sascha is correct: there are small MIP problems which are well known to be very hard to solve. The paper is a good reference. Additional suggestions: set read datacheck parameter to 2 to rule out potential numerical issues Try non-standard mip emphasis values, 3 may give good results if best bound is stuck, and 4 sometimes gives very good results. MIP emphasis 5 may help if you set a time limit as it tries to compute good solutions early.

docplex code:

mdl.parameters.read.datacheck =2
mdl.parameters.emphasis.mip = 3 (or 4, 5)

If this does not work, please publish a larger extract of the CPLEX log, starting a the very beginning.

Answer 2

From the full log, I can see many warnings of this type from the modeling assistant (read.datacheck=2)

CPLEX Warning  1008: Detected righthand side <= CPX_MINBOUND at constraint 'c9198'.

This means that the constraint's right-hand side is too small. CPLEX comducts all calculartions in floating-point numbers, which are by essence non-exact; therefore all calculations are subject to tolerances. The tolerance for constraint satisfaction is 1e-6, in other terms a constraint such as

X + 2 Y >= 3

can be satisfied by a CPLEX solution with a tolerance marging of 1e-6

To put it simply: do not use coefficients or constants below 1e-6 (in absolute value) . Using such tiny values can only perturbate cplex in useless numerical issues. Either use plain 0 or remove the constraint. For example, a constraint like:

  X + 2 Y <= 1e-12

should be written

  X + 2 Y <= 0

Same rule for coefficients: avoid coefficients below 1e-7. If your data contains them, either filter them out or scale the data. The recommendation is that in the whole model, the orders of magnitude for coefficients should not span more than 1e+6 , in other terms, the ratio of the biggest to the smallest should stay under 1e+7; otherwise numerical trouble is to be expected.

As for parameters, I see you are using strong branching. Use it only if it really helps as it can be extremely expensive in CPU. I also noticed you restrained threads to 2, why not let cplex use as many as possible?

To summarize, first fix those tiny coefficients, then try without strong branching and full threads

Answer 3

From your new log, I see all data issues have been fixed, which is good. Still, the mip search is stuck, with best bound not moving. Your problem has mostly binary variables, so highly combinatorial; such problems can be very hard to solve. Still, a few general suggestions:

• Try without the mip strategy nodeselect 0, I'm really not convinced this actually helps.
• same for mip strategy file: try removing it
• As your best bound is stuck, did you try with mip emphasis=3, supposed to work on this best bound

In addition you might try the "polishing" heuristic, it sometimes gives good result. To switch to polishing after a given gap value use "set mip strategy polishafter mipgap 0.40" in cplex interactive, this will switch polishing when the 40% gap threshold is passed.

If nothing works, then maybe there are issues in the model formulation itself. One idea is to set a trivial objective (say some variable, or sum of binaries) and see whther the situation improves. If yes, the issue is in finding better solution, if no, the issue is in finding feasible solutions. IN the latter case, the feasibility pump (mip strategy fpheur 1 or 2) might help.