-1

Is there any out of the box newton like solver that allows bounds on the input parameters (and a supplied expression for the jacobian and hessian).

I am basically exactly looking for "trust-constr" in SciPy, except that one does not seem to work at all. While it accepts bounds as an argument it seems to just ignore them. See also this question Scipy: How can I use Bounds with trust-constr? about this issue.

It seems that the failure of trust-constr to take into account bounds is not completely general. It would probably be nicest to show an instance with something standard like the rosen function but my initial try did not work. So here is a simple counter example

def fake_function(x):
    if x[0]==0.523 and x[1]==1.43:
        return -0.0318285
    print("value at x not available")

def fake_grad(x):
    return [9.21296, -1.98147]

def fake_hess(x):
    return [[-467.451, -98.9485], [-98.9485, 28.6649]]

scipy.optimize.minimize(fake_function, [0.523, 1.43], method='trust-constr', jac=fake_grad, hess=fake_hess,
                                  bounds=[(0.5,0.55),(0.5,2)],
                                  options={'gtol': 1e-5, 'disp': True})

Obviously this optimization doesn't make sense after the first step because I defined the function in by its value at the initial point. However, it is enough to show that at the very first step trust-constr will ignore the bounds and ask for a point outside them. (Now clarified to show that the fake_function is a well defined twice differentiable function that is too big to include in the minimal example. We only need the values at the initial point however to see the algorithm misbehave.)

Also changing the bounds a bit I realize that it is not completely ignoring them. It chooses different points based on the bounds however it does not mind stepping outside the bounds which seems to be a terrible feature.

Kvothe
  • 233
  • 1
  • 8
  • Can you provide a minimal example that fails for you? The example in your linked post works for me with scipy 1.7.0, i.e. I obtain a solution satisfying the bounds. And to answer your question: Yes, the 'trust-constr' method can handle bounds and accepts Jacobians hand Hessians both for the objective and the constraints. – joni Jun 24 '21 at 07:10
  • @joni thanks for pointing out that the problem at least does not seem to be completely general. Sometimes it does seem to work. I will add a minimal counter example as soon as I construct it. It definitely happily steps outside the constraints in my actual problem but I now see that in a very simple rosen example I can't easily reproduce this. – Kvothe Jun 24 '21 at 10:31
  • @joni, I added a minimal example where the algorithm immediately steps outside the bounds. (I defined my functions in a constant way with the values from my actual problem, so the optimization doesn't make much sense after the first step but by then my point has already been shown.) – Kvothe Jun 24 '21 at 10:44
  • Since both the gradient and the hessian are wrong, I don't see how this should be a valid counter-example. Your objective is constant, hence the gradient and the hessian are zero. – joni Jun 24 '21 at 11:01
  • @joni, no you misunderstand the point. All values are the correct ones at the point [0.523, 1.43]. (Perhaps I can clarify this by defining it only at that point). I just can't include the whole function since it would be way to big to include in a "minimal" example. – Kvothe Jun 24 '21 at 11:36
  • Note that the failure occurs already before any function value is demanded at a second point and so it does not matter that this one is incorrect. We already know that the algorithm failed. – Kvothe Jun 24 '21 at 11:38
  • @joni, I emphasized that these are simply the initial values at the original point of a more complicated twice differentiable function. And that the values at the original point are already enough to see that it ignores the bounds. Do you understand? – Kvothe Jun 24 '21 at 11:50

1 Answers1

2

Despite the (imho) terrible setup for demonstrating your problem, consider explicitly enforcing feasiblity (docs):

Code

import numpy as np
from scipy.optimize import minimize, Bounds

def fake_function(x):
    print('EVAL @: ', x)

    if x[0]==0.523 and x[1]==1.43:
        return -0.0318285
    print("value at x not available")

def fake_grad(x):
    return [9.21296, -1.98147]

def fake_hess(x):
    return [[-467.451, -98.9485], [-98.9485, 28.6649]]

# !!!
bounds_explicit_feas = Bounds(np.array([0.5, 0.5]), np.array([0.55, 2.0]), keep_feasible = True)
bounds_original = [(0.5,0.55),(0.5,2)]
# !!!    

minimize(fake_function, [0.523, 1.43], method='trust-constr', jac=fake_grad, hess=fake_hess,
                                  bounds=bounds_explicit_feas,
                                  options={'gtol': 1e-5, 'disp': True})

Original Bounds: Output

EVAL @:  [0.523 1.43 ]
EVAL @:  [0.19537876 1.56399192]
value at x not available

Explicit Feasible: Output

EVAL @:  [0.523 1.43 ]
EVAL @:  [0.52256323 1.49080832]
value at x not available
sascha
  • 32,238
  • 6
  • 68
  • 110