1

I am trying to implement a nonlinear optimisation with 16 linear inequality constraints and one linear equality constraint by using mystic for a problem with 100 variables. When I use the linear_symbolic module to create the constraints, the execution gets stuck. I find not other obvious way to add the constraints.

Scipy's minimize module produces bad solutions for objective functions that are not of a quadratic type (my objective is relative entropy) and when the starting values are not very close to the optimal solution so I had to abandon it. This is actually a point that that Mike McKerns makes when motivating mystic so I was very happy when I bumped into it.

I have tried the following two lines:

cf = mystic.symbolic.linear_symbolic(np.ones(100),[1.],G,h) 
cons = mystic.symbolic.generate_constraint(mystic.symbolic.generate_solvers(mystic.symbolic.simplify(cf)))

where G is a 16x100 matrix of coefficients and h is a 16-dimensional vector containing the constants on the right-hand side of the inequalities.

The code executes relatively fast the first line above but it never goes through the second one (my computer crashes after a long wait).

Iñaki Rodríguez Longarela
  • 11
  • 2

1 Answers1

0

Similar question/answer to Issues regarding constraint's setting via mystic package.

Simplify with inequalities takes a long while as it cycles through all possible sign-flips, and assumes the worst-case for isolating one variable on the left-hand side of each equation. Doing the following will isolate one equation at a time, which then you could (in theory) couple with the constraints keyword "join", or as below:

>>> import mystic      
>>> import numpy as np
>>> G = np.random.random((16,30)) * 100 
>>> h = np.random.random((16,)) * 1000
>>> c = mystic.symbolic.linear_symbolic(np.ones(30),[1.],G,h)
>>> c = c.strip().split('\n')
>>> cs = '\n'.join(mystic.symbolic.simplify(ci, target='x{i}'.format(i=i)) for i,ci in enumerate(c))
>>> cons = mystic.symbolic.generate_constraint(mystic.symbolic.generate_solvers(cs))

The issue, apparently, is that while solving for 30 variables works generally, going to 100 per line is too much stress on the memory of the computer used. So, literally, the issue is the code to simplify the inequalities is not as efficient as it could be for simple cases, and becomes slow once it start pegging the memory. I'll have to look into reducing the memory footprint... but you may be able to workaround it for your case by doing some more manual simplification (all that needs to be done is to isolate a single variable on each left-hand side of each equation).

In the case above, it's fast for 10 variables, but takes a minute or so for 30. For the case where there's clearly a single variable that's only used once, and thus easy to isolate, it shouldn't take that long. I'm sure that with some minor cleaning, I can reduce the memory requirements of the function. I've created a ticket for this issue: https://github.com/uqfoundation/mystic/issues/113.

Mike McKerns
  • 33,715
  • 8
  • 119
  • 139
  • Thanks a lot, Mike! I tried your code but it still fails (as you say it would given the size of my problem). Following your advise regarding manual simplification, I wonder if it would work to add slack variables to turn the inequalities into equalities and then since each one of those slack variables only enters once, it would be trivial to put them on the left-hand side.The only thing left would be to solve for one variable in the equality and substitute back into the equalities (previously inequalities).Will the fact that the slack variables do not enter the objective function be a problem? – Iñaki Rodríguez Longarela May 16 '19 at 20:34
  • Two things: (1) I convert the inequalities to equalities then use sympy to solve the equalities... then turn them back to inequalities, with all possible sign flips. Slack variables may help, I'll look at that. (2) After further investigation, the real culprit is `sympy`. So, in timing a `simplify` of one line with 30 variables, I get an answer in about 12 sec. 0.6 of that is an import of `sympy`, and over 11 sec of it is the symbolic solve in `sympy`. So, it's not going to be as simple as I thought to make improvements, but at least I've identified the culprit. – Mike McKerns May 17 '19 at 13:30
  • Slack variables are reasonable... see: https://github.com/uqfoundation/mystic/blob/master/examples2/slack_variable.py – Mike McKerns May 17 '19 at 13:34
  • 1
    Thanks again, Mike!! I have managed to do the simplify for the inequalities by using slack variables and inverting the submatrix of coefficients multiplying the variables chosen to solve for. This works fine if the matrix of coefficients is full rank (maybe a tip to speed up the simplify?). Anyway, my problem now is that after I feed the diffev2 solver with all the right ingredients, it seems to be stuck. It does not do any function evaluation and if I give a value to maxiter, it reaches its maximum without evaluating the objective function once. What could this be due to? – Iñaki Rodríguez Longarela May 18 '19 at 15:13
  • Cool. I'm first going to see what I can do to speed up the code block that interacts with `sympy`. That's where all the issues are... but if what you suggest impacts the ability of `sympy` to solve the problem quickly, then it very well should help. In terms of the optimization, I'm going to guess it's probably taking time trying to impose the constraints. My suggestion is to try NelderMead first, as it's the cheapest in terms of per-iteration cost. When you see everything "works" there, then you can move to a better solver (better to post this follow-on discussion on mystic's GitHub page). – Mike McKerns May 18 '19 at 16:25
  • I just pushed some changes to GitHub that significantly decrease the time required for the symbolic simplification/solve. I'll cut a release in the next week or so. It's a significant improvement in speed -- thanks for posting about this. – Mike McKerns May 19 '19 at 19:57
  • Sorry, saw this just now. Great! Thank you, Mike! – Iñaki Rodríguez Longarela Jul 04 '19 at 12:38