Find function minimum for discrete variables w/ bounds and constraints

Question

I'm trying to find a (relatively) fast way to minimise a function on the set of natural numbers given constraints and bounds. I know the mathematical form of the function and its constraints, so a brute force approach seems slow and not very elegant. What's the best way to go about this problem?

Basically I'm trying to generalise from a functional minimisation on the real numbers using scipy.optimize.minimize to one on the natural numbers. (I'm aware that this is a lot harder)

To make things easy. I'm thinking of something like this example:

from scipy import optimize

x0 = [0,0]
cons = ({'type': 'ineq', 'fun': lambda x: 0.4 - x[1]})
bounds = ((0,None),(0,None))
fun = lambda x: (x[0]-1.5)**2 + (x[1]-0.5)**2 + 3

res = optimize.minimize(fun, x0=x0, bounds=bounds, constraints=cons)
print(res)

Put differently, I'm looking to add constraints and bounds on

fun = lambda x: (x[0]-1.5)**2 + (x[1]-0.5)**2 + 3
xr = [(0,0),(0,1),(0,2),(0,3),(1,0),(1,1),(1,2),(1,3),(2,0),(2,1),(2,2),(2,3),(3,0),(3,1),(3,2),(3,3),]
min_idx = min(xr, key=fun)
min_val = fun(min_idx)

print(min_idx,min_val)

(I know I could impose them by excluding those values from xr, but that seems very little elegant and less practical for the actual thing I have in mind)

So I'm expecting there to be some sort of different minimiser like scipy.optimize.basinhopping or something from mystic to do the trick? Any suggestions?

Without knowing more about the minimization problem, I would try branch and bound https://en.wikipedia.org/wiki/Branch_and_bound — Juan Carlos Ramirez, Jul 26 '19 at 15:44

score 3 · Answer 1 · answered Jul 28 '19 at 22:13

I modified your problem slightly, to make it a very little bit harder...

  Minimize:
            f(x) = (x0 - 1.5)**2 + (x1 - 0.5)**2 + 3

  Where: 
            0 =< x0
            0 =< x1 < 4.1 - x0
            x0,x1 are integers

With mystic, you can solve the problem relatively directly:

>>> def objective(x):
...     return (x[0] - 1.5)**2 + (x[1] - 0.5)**2 + 3
... 
>>> bounds = [(0,None)]*2
>>> equations = """
... x1 < 4.1 - x0
... """
>>> 
>>> from mystic.symbolic import generate_penalty, generate_conditions
>>> pf = generate_penalty(generate_conditions(equations))
>>> 
>>> from mystic.constraints import integers
>>> 
>>> @integers()
... def round(x):
...   return x
... 
>>> from mystic.solvers import diffev2
>>> from mystic.monitors import VerboseMonitor
>>> 
>>> result = diffev2(objective, x0=bounds, bounds=bounds, penalty=pf, constraints=round, npop=40, gtol=100, disp=True, full_output=True, itermon=VerboseMonitor())
Generation 0 has ChiSquare: 9364567.500000
Generation 10 has ChiSquare: 7021.500000
Generation 20 has ChiSquare: 3.500000
Generation 30 has ChiSquare: 3.500000
Generation 40 has ChiSquare: 3.500000
Generation 50 has ChiSquare: 3.500000
Generation 60 has ChiSquare: 3.500000
STOP("ChangeOverGeneration with {'tolerance': 0.005, 'generations': 100}")
Optimization terminated successfully.
         Current function value: 3.500000
         Iterations: 67
         Function evaluations: 2400
>>> 
>>> print(result[0])
[2. 0.]

Mystic also has logical operators to couple constraints and penalties, and constraints that restrict potential solutions to discrete values, unique values, etc.

Hey! That's pretty cool! Thank a lot! I was wondering: Since you use "round", the solver will waste a lot of time checking equivalent numbers (say 1.6 and 1.7) for the parameters. I am assuming there is no way to save the solver this extra exercise right? And on a side-note: Would it make a difference perofrmance-wise to use "int" instead of "round"? — Spectral Confusion, Jul 30 '19 at 09:10
Not sure what's more performant, `round` or `int`. Yes, you can actually save the solver easily, as it's serializable. You can use the class interface to the solvers, instead of the function interface -- so that's the `SaveSolver` method in the `DifferentialEvolutionSolver2` class (instead of `diffev2`) — Mike McKerns, Jul 30 '19 at 10:34
in `mystic`, a constraint is a mapping/filtering of search space (i.e. a coordinate transformation), while a penalty is an addition to the cost when constraints are violated. So constraints are "hard" and penalties are "soft". Using `round` as a constraint, will basically force the solver to only search the space of integers. — Mike McKerns, Jul 30 '19 at 10:38
Okay, thanks! That clears it up. Thanks so much for putting together the example! — Spectral Confusion, Aug 02 '19 at 13:01

Find function minimum for discrete variables w/ bounds and constraints

1 Answers1