Solver 'CPLEX' failed. Suggestion of parameters

Question

I am trying to use cplex to solve a LP optimization problem. (In python using cvxpy)

Depending on the constraints I place on the problem, the cplex solver sometimes fails to find a solution. I'd like some intuitions on how to read the output of the solver when the verbose=True is provided.

For example, I get this:

Version identifier: 12.10.0.0 | 2019-11-26 | 843d4de
CPXPARAM_Read_DataCheck                          1
CPXPARAM_Preprocessing_QCPDuals                  2
Found incumbent of value 0.000000 after 0.00 sec. (0.08 ticks)
Tried aggregator 2 times.
MIP Presolve eliminated 4726 rows and 353 columns.
MIP Presolve modified 1008 coefficients.
Aggregator did 3 substitutions.
Reduced MIP has 1669 rows, 1670 columns, and 4505 nonzeros.
Reduced MIP has 1168 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.04 sec. (5.22 ticks)
Probing fixed 0 vars, tightened 2 bounds.
Probing time = 0.01 sec. (0.85 ticks)
Tried aggregator 1 time.
Detecting symmetries...
MIP Presolve modified 2 coefficients.
Reduced MIP has 1669 rows, 1670 columns, and 4505 nonzeros.
Reduced MIP has 1168 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.03 sec. (3.10 ticks)
Probing time = 0.01 sec. (0.90 ticks)
Clique table members: 2327.
MIP emphasis: balance optimality and feasibility.
MIP search method: dynamic search.
Parallel mode: deterministic, using up to 4 threads.
Root relaxation solution time = 0.01 sec. (5.82 ticks)

        Nodes                                         Cuts/
   Node  Left     Objective  IInf  Best Integer    Best Bound    ItCnt     Gap

*     0+    0                            0.0000  -325889.6000              --- 
      0     0   -23545.7611   333        0.0000   -23545.7611      723     --- 
*     0+    0                       -15240.4400   -23545.7611            54.50%
      0     0   -21360.6115   333   -15240.4400     Cuts: 333     1063   40.16%
*     0+    0                       -20698.8800   -21360.6115             3.20%
      0     0   -21120.8531   333   -20698.8800     Cuts: 268     1232    2.04%
*     0+    0                       -20751.6200   -21120.8531             1.78%
      0     0   -21057.6246   333   -20751.6200      Cuts: 92     1322    1.47%
*     0+    0                       -20843.7200   -21057.6246             1.03%
      0     0   -21023.5211   333   -20843.7200      Cuts: 90     1397    0.86%
      0     0   -20983.6905   333   -20843.7200      Cuts: 57     1440    0.67%
*     0+    0                       -20877.4400   -20983.6905             0.51%
Detecting symmetries...
      0     0   -20972.8355   333   -20877.4400      Cuts: 13     1449    0.46%
*     0+    0                       -20878.2600   -20972.8355             0.45%
      0     0   -20970.0341   333   -20878.2600      Cuts: 19     1460    0.44%
      0     0   -20969.6020   333   -20878.2600      Cuts: 10     1471    0.44%
      0     0   -20969.2988   333   -20878.2600    MIRcuts: 4     1476    0.44%
      0     0   -20959.2311   333   -20878.2600       Cuts: 9     1483    0.39%
*     0+    0                       -20935.7200   -20959.2311             0.11%
      0     0   -20958.0881   333   -20935.7200      Cuts: 15     1500    0.11%
*     0+    0                       -20935.7200   -20958.0881             0.11%
Detecting symmetries...

Repeating presolve.
Tried aggregator 2 times.
MIP Presolve eliminated 1039 rows and 925 columns.
MIP Presolve modified 62 coefficients.
Aggregator did 102 substitutions.
Reduced MIP has 528 rows, 639 columns, and 1468 nonzeros.
Reduced MIP has 526 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.04 sec. (2.78 ticks)
Probing fixed 0 vars, tightened 18 bounds.
Probing time = 0.00 sec. (0.36 ticks)
Tried aggregator 2 times.
MIP Presolve eliminated 241 rows and 308 columns.
MIP Presolve modified 23 coefficients.
Aggregator did 2 substitutions.
Reduced MIP has 285 rows, 329 columns, and 792 nonzeros.
Reduced MIP has 261 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.01 sec. (0.68 ticks)
Probing fixed 0 vars, tightened 1 bounds.
Probing time = 0.01 sec. (0.18 ticks)
Tried aggregator 1 time.
Detecting symmetries...
MIP Presolve modified 1 coefficients.
Reduced MIP has 285 rows, 329 columns, and 792 nonzeros.
Reduced MIP has 261 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.03 sec. (0.58 ticks)
Represolve time = 0.18 sec. (19.83 ticks)
Probing time = 0.00 sec. (0.18 ticks)
Clique table members: 547.
MIP emphasis: balance optimality and feasibility.
MIP search method: dynamic search.
Parallel mode: deterministic, using up to 4 threads.
Root relaxation solution time = 0.00 sec. (1.32 ticks)

        Nodes                                         Cuts/
   Node  Left     Objective  IInf  Best Integer    Best Bound    ItCnt     Gap

*     0+    0                       -20935.7200   -20949.2232             0.06%
      0     0   -20950.6138    31   -20935.7200   -20949.2232     1635    0.06%
      0     0   -20946.9943    31   -20935.7200      Cuts: 27     1650    0.05%
      0     0   -20946.9195    31   -20935.7200    MIRcuts: 4     1653    0.05%
      0     0   -20945.8979    31   -20935.7200      Cuts: 11     1658    0.05%
      0     0   -20945.8979    31   -20935.7200   Flowcuts: 2     1659    0.05%
      0     0   -20945.8979    31   -20935.7200   Flowcuts: 1     1663    0.05%
Detecting symmetries...

Clique cuts applied:  10
Implied bound cuts applied:  3
Flow cuts applied:  35
Mixed integer rounding cuts applied:  37
Lift and project cuts applied:  3
Gomory fractional cuts applied:  1

Root node processing (before b&c):
  Real time             =    1.34 sec. (232.86 ticks)
Parallel b&c, 4 threads:
  Real time             =    0.00 sec. (0.00 ticks)
  Sync time (average)   =    0.00 sec.
  Wait time (average)   =    0.00 sec.
                          ------------
Total (root+branch&cut) =    1.34 sec. (232.86 ticks)

From a given run. I know from this how to pass parameters through cvxpy to cplex, but reading the output of the solver does not help me determine if the solver fails due to a memory issue, or a numerical issue or anything of that sort and adapt the parameters consequently. I'd also like to point out that the set of constraints I am using is large (can reach 34 for every data point) but that the solver still fails on very small data frames (only 24 points)

Any suggestions/materials that could help me?

Thanks a lot!

The easiest thing is to export the model to a file (preferably SAV) and then investigate using the interactive CPLEX optimizer. There you can run the conflict refiner to find a conflict. — Daniel Junglas, Apr 14 '20 at 13:11
You don't say how soon the solver fails. Does it fail very quickly, or does it fail after a (long) time? What do you get from the log file - the excerpt that you give above suggests that the problem is feasible and the solver has found a solution. Do you not get any log when the solver fails? Can you post more info about what happens when the solver fails. — TimChippingtonDerrick, Apr 14 '20 at 19:30
It fails relatively quickly (just a few seconds) with an exception raised from `~/.virtualenvs/tomorrow/lib/python3.7/site-packages/cvxpy/problems/problem.py in unpack_results(self, solution, chain, inverse_data) 716 raise error.SolverError( 717 "Solver '%s' failed. " % chain.solver.name() + --> 718 "Try another solver, or solve with verbose=True for more " 719 "information.") 720 self.unpack(solution)` triggered by the solver returning the status `solver_error` — pierresegonne, Apr 15 '20 at 07:04

score 0 · Answer 1 · answered Apr 14 '20 at 13:11

You could use status. Let me use the bus example.

# Import packages.
import cvxpy as cp


# Define and solve the CVXPY problem.
nbBus40 = cp.Variable(integer=True)
nbBus30 = cp.Variable( integer=True)
cost = 500*nbBus40+400*nbBus30
prob = cp.Problem(cp.Minimize(cost),[40*nbBus40+30*nbBus30>=300,
                                     nbBus40>=0,nbBus30>=0
                                     ])

prob.solve(solver=cp.CPLEX,verbose=True)

print("status = ",prob.status)

# Print result.
print("\nThe minimal cost is", prob.value)

print("number buses 40 seats = ",nbBus40.value)
print("number buses 30 seats = ",nbBus30.value)

gives

status =  optimal

The minimal cost is 3800.0
number buses 40 seats =  6.0
number buses 30 seats =  2.0

whereas if we change the constraint to

prob = cp.Problem(cp.Minimize(cost),[40*nbBus40+30*nbBus30>=300,
                                     nbBus40>=0,nbBus30>=0,nbBus40<=2,nbBus30<=2
                                     ])

then we get

status =  infeasible

The minimal cost is inf
number buses 40 seats =  None
number buses 30 seats =  None

Thanks for your example Alex, but I can't access the status as the solver fails before returning any status. What I am looking for is an efficient way to investigate what makes the solver fail, and if tweaking the parameters might allow to find a solution — pierresegonne, Apr 14 '20 at 13:17
Then try prob.solve(solver=cp.CPLEX,verbose=True,cplex_filename="c://temp//cvx.sav") in order to generate sav file that you could debug with interactive cplex — Alex Fleischer, Apr 14 '20 at 13:35

Solver 'CPLEX' failed. Suggestion of parameters

1 Answers1