scipy.minimize does not seem to adhere to constraints

Question

scipy.minimize does not seem to adhere to constraints. Here is a simple example where the constraint is for preventing a negative argument in the logarithm, but the minimize function does not adhere :

import math
from scipy.optimize import minimize

def obj(x,b):
    print "obj x",x
    return math.log(x-b)

def constr(x,b):
    print "constr x",x
    return x-b

x=3.1
b=3
a=minimize(obj,x,args=(b),constraints={'type': 'ineq', 'fun':constr,'args':[b]})

the output is:

constr x [ 3.1]
obj x [ 3.1]
constr x [ 3.1]
obj x [ 3.1]
obj x [ 3.10000001]
constr x [ 3.1]
constr x [ 3.10000001]
obj x [ 3.]
Traceback (most recent call last):
File "scipy_minimize_constraints.py", line 19, in 
a=minimize(obj,x,args=(b),constraints={'type': 'ineq', 'fun':constr,'args':[b]})
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/_minimize.py", line 495, in minimize
constraints, callback=callback, **options)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/slsqp.py", line 378, in _minimize_slsqp
fx = func(x)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 292, in function_wrapper
return function(*(wrapper_args + args))
File "scipy_minimize_constraints.py", line 9, in obj
return math.log(x-b)
ValueError: math domain error

python2.7 Scipy version 1.0.0

Am I doing something wrong?

score 3 · Answer 1 · answered Jun 01 '18 at 10:50

Constraint

def constr(x,b):
    return x-b

results in x-b >= 0 (nonnegative), but allows x-b = 0. Then log(x-b) is undefined if x-b = 0. You would need to introduce some epsilon like:

eps = 1e-12
def constr(x,b):
    return x-b-eps

which won't throw your error.

But there might be a more important problem: i really don't think the solver (SLSQP here) guarantees it's iterates to be feasible! There might be cases where this will be problematic. In your simple example, the way to go is to transform your constraint to bounds. That's of course less expressive (not always possible; but for your tiny example it is), but those bounds are respected in the iterates!

inferred_lb = b + eps
a = minimize(obj, x, args=(b,), bounds=[(inferred_lb, None)])

(And of course: don't use minimize for single-dimension optimization. There is minimize_scalar.)

Jay Calamari · Answer 2 · 2018-06-01T18:25:34.727

I don't know exactly how scipy's minimize works, but I think you're right that it doesn't follow the constraints.

If, in obj(x,b), I try printing x - b, I get negative output for the last loop when it throws the error.

this isn't totally surprising though. the function you're minimizing is essentially log(z) with the constraint z > 0. that's not going to converge very slowly... :p

EDIT: setting the keyword parameter "tol" (error tolerance) to around 0.1 for the minimizer gets rid of the error ( as in minimize(..., tol=0.1)). I still don't know why it's so significant in the first place.

EDIT (since I still can't comment): @sascha, that makes sense, but why does the function even evaluate negative values of x - b? Does the minimize function "test" outside of its constraints just in order to better optimize the solution within it? (If I should ask this as a new question, let me know.)

scipy.minimize does not seem to adhere to constraints

2 Answers2