1

I am trying some performance experimentation using the CPLEX Python API and found that most of the CPLEX row or column updates are via sparse pair or sparse triplets. On large matrices ( millions of columns), it just gets too slow.

I am looking for more efficient ways (equivalent of int CPXaddcols(CPXENVptr env,CPXLPptr lp,int ccnt,int nzcnt,double *obj,int *cmatbeg,int *cmatind,double *cmatval, double *lb, double *ub,char **colname) from Callable Library)

I found the methods to be in the cplex._internal._procedural, but could not really call it successfully.

Anyone used CPLEX Python for large scale LP? Any one used the addcols(env, lp, ccnt, nzcnt, obj, cmat, lb, ub, colname, enc=default_encoding) method in python cplex?

Any help is highly appreciated.

Thanks

Code

import cplex 

import os

WORKDIR = "data1"

filename = "matrix1.prefix.LP"

os.chdir(WORKDIR)

problem = cplex.Cplex(filename)

theEnv = problem._env

theLP = problem._lp

returnCols = cplex._proc.getcols(theEnv,  theLP, 0,10)

print( returnCols)

Output

File "C:/TSWork/Misc/Calendar_Year_2018/Experiments/MtrxDnst/CPX_Proc_V1.py", line 15, in returnCols = cplex._proc.getcols(theEnv, theLP, 0,10)

File "C:\Software\Anaconda3\envs\cplex\lib\site-packages\cplex_internal_procedural.py", line 861, in getcols status = CR.CPXXgetcols(env, lp, inout_list)

TypeError: in method 'CPXXgetcols', argument 1 of type 'CPXCENVptr'

Ioannis
  • 5,238
  • 2
  • 19
  • 31
Tushar Shekhar
  • 11
  • 2
  • 1
    Hi there! Welcome to Stack Overflow :) It's VERY helpful to folks trying to answer your question if you 1) post some code you're trying using the syntax highlight features, 2) the actual error message you're getting, and 3) the desired result! This will get you the best quality help and answers. Optionally, post a few different things you've tried and this will help us rule out troubleshooting options! – snakes_on_a_keyboard Apr 12 '18 at 15:54
  • Can you share some code that shows how you are adding the columns using the CPLEX Python API? Are you adding them in batches or one at a time? Are you using names or indices? Adding in batches and using indices should yield far better performance. – rkersh Apr 12 '18 at 21:45
  • I added the code and the output in the question. The question that I need answer to is : I see a lot of procedural methods that correspond to C callable library, but I am not able to use them. I am assuming these functions will be much faster than the stock cplex.Cplex.variables.add() functions ( sorry, cannot shake my c callable library experience). – Tushar Shekhar Apr 13 '18 at 06:24
  • Have a look at lpex6.py, which populates the LP by columns. It uses the `cplex.cplex.variables.add()` method. – Ioannis Apr 13 '18 at 08:00
  • @loannis , lpex6.py still does not use the 'real sparse coefficient representation' that is made possible in the method like _preaddrows(env, lp, rhs, sense, rmatbeg, rmatind, rmatval, names, enc=default_encoding)_ . In my question, I have given an example of using cplex._proc.getcols(theEnv, theLP, 0,10), but I can't seem to get the correct values for the envPtr and lpPtr. If you can help me correct that problem, I think I can start using other functions from the cplex._proc that is faster. Thanks. – Tushar Shekhar Apr 13 '18 at 09:36
  • Using the code in `cplex._internal._procedural` is not supported and is subject to change at any time. In my comment above, I'm asking you to show the code using the public API that you think is slow. – rkersh Apr 13 '18 at 14:56
  • Hi rkersh, I have a cplex prob that I need to mod and solve. I try to delete rows or cols and solve, but *origProblem.linear_constraints.delete(set_rows_to_remove)* and *origProblem.columns.delete(set_cols_to_remove)* effectively hangs on a problem where *num_columns=1880278* and *num_rows=513167*. Another option I have is to create a problem from scratch by adding the rows and columns to a blank problem, that's why I was hoping to use some more efficient way of constructing a problem. Either of the ways work for me (a) either create a problem fast or (b) delete components of the problem fast. – Tushar Shekhar Apr 16 '18 at 04:51
  • @TusharShekhar: is `set_rows_to_remove` a sequence? Are the indices contiguous? If possible, the most efficient way to delete is to use ranges (i.e., `delete(begin, end)`). – rkersh Apr 16 '18 at 16:50
  • @rkersh, the set_rows_to_remove is a set of indices and is not contiguous. Can try to extract some contiguous series and use this, thnx for the suggestion. But I still can't solve my problem. Can I ask how you do it (I am assuming you use cplex python for practical size LP) – Tushar Shekhar Apr 17 '18 at 02:47

0 Answers0