0

I want to solve an optimal transport for source and destination nodes of sizes 42000 and 18000 respectively. I know that Scipy Linear Programming Module now includes HIGHS so it should be pretty efficient to use. However, I have been running the program for over 15 hours and it has not found a result. I am representing my constraints matrix as a sparse matrix (CSR). There are other commercial options I could use like CPLEX but I am curious to see if Scipy is able to scale to a linear program of this size. My code can be found at the Pastebin link located below. My problem setting is finding optimal transport between different classes in MNIST. If Scipy should scale to linear programs of this size, is there anything I can do to improve the runtime?

Pastebin link: https://pastebin.com/rDTh4V49
indispinablenorm
  • 55
  • 5
  • If the problem is dense (i.e. all links exist), we end up with an LP with 756,000,000 variables. This is very large. If possible make use of sparsity in the underlying graph. Also, make sure you are not running out of memory. – Erwin Kalvelagen Apr 04 '22 at 19:01
  • @ErwinKalvelagen Given that it is an optimal transport problem, it is a fully connected graph from the source to the destination nodes. However, the constraints matrix is sparse so it is being fed as a scipy.sprarse.CSR matrix. Is there anything else you recommend I should try? – indispinablenorm Apr 04 '22 at 19:54

0 Answers0