0

my objective is to conduct quadratic curve fitting according to y=a*x**2 + b *x+cwith bounds on two variables variables a<0,c<=0 and a constraint 2*a*x+b >=0. In short, I want the fit to be monotonic increasing function in first quadrant with y(x=0) <= 0. My sample code consists of three trials in using scipy.optimize.minimize function:

(1) I don't specify jacobian: works perfect on regular data.

(2) specify jacobian: is not able to predict even on regular data: some issues with jacobian definition.

(3) apply constraint: constrain definition is raising error.

Can I seek your attention to make this program run ?

import numpy as np
from scipy.optimize import minimize

# scattr-irregular
#xip=np.array([ 0.02237461, 0.0983837 , 0.25707382, 0.56959641, 1.33419197, 4.95835927])
#yip=np.array([0.20085822, 0.23583258, 0.28996988, 0.36350284, 0.47981232, 0.67602165] )
#scatter- regular
xip=np.array([2.12E-01,3.43E-01,5.01E-01,7.01E-01,9.73E-01,1.15E+00,1.34E+00,1.41E+00])
yip=np.array([5.94E-02,1.28E-01,2.05E-01,2.90E-01,3.86E-01,4.34E-01,4.78E-01,4.93E-01])

import  matplotlib.pyplot as plt
plt.scatter(xip,yip, label = 'scatter')

xipfit=np.linspace(0, max(xip), 200)

#minimize.scipy
def func(abc,x,y):
    """ Objective function """
    return ((abc[0]*x**2+abc[1]*x**1+abc[2]-y)**2).sum()

def func_deriv(abc, x,y):
    """ Derivative of objective function """
    dfda = 2*(abc[0]*x**2+abc[1]*x**1+abc[2]-y)*x**2
    dfdb = 2*(abc[0]*x**2+abc[1]*x**1+abc[2]-y)*x**1
    dfdc = 2*(abc[0]*x**2+abc[1]*x**1+abc[2]-y)* 1
    return np.array([ dfda.sum(), dfdb.sum(), dfdc.sum() ])  

#2*a*x+b >=0    
cons = ({'type': 'ineq',
         'fun' : lambda abc: np.array( [ 2*abc[0]*xip.sum()+abc[1] ]) ,
         'jac' : lambda abc: np.array([2*xip.sum(),  1,  0])\
         })    
######################################################    
###1. wO jacobian input; works for specific data.    
coeffs_WOjac= minimize(func, x0=[-1.0,1.0,-1], args=(xip,yip),\
                jac=False,\
                method='SLSQP',\
                bounds=((-1e6,0),(0,1e6),(-1e6,0)),\
                options={'disp': True})
yipfit_WOjac = coeffs_WOjac.x[0]*xipfit**2\
                +coeffs_WOjac.x[1]*xipfit \
                +coeffs_WOjac.x[2]
print(coeffs_WOjac)                
plt.plot(xipfit,yipfit_WOjac,label='WOjac', linewidth=4)
######################################################                
###2. wit jacobian input: fit is not correct. jacobain correct ?             
coeffs_Wjac= minimize(func, x0=[-1.0,1.0,-1], args=(xip,yip),\
                jac=func_deriv,\
                method='SLSQP', \
                bounds=((-1e6,0),(-1e6,1e6),(-1e6,0)),\
                options={'disp': True})
yipfit_Wjac = coeffs_Wjac.x[0]*xipfit**2\
                +coeffs_Wjac.x[1]*xipfit \
                +coeffs_Wjac.x[2]
print(coeffs_Wjac)
plt.plot(xipfit,yipfit_Wjac,label='Wjac', color= 'red',linewidth=1.5)
######################################################                
###3. wO jacobian and constraints  
coeffs_WOjac_constr = minimize(func, x0=[-1.0,1.0,-1], args=(xip,yip),\
                jac=False,\
                constraints=cons,\
                method='SLSQP', 
                bounds=((-1e6,0),(-1e6,1e6),(-1e6,0)),\
                options={'disp': True})                
yipfit_WOjac_constr = coeffs_WOjac_constr.x[0]*xipfit**2\
                +coeffs_WOjac_constr.x[1]*xipfit \
                +coeffs_WOjac_constr.x[2]

plt.plot(xipfit,yipfit_WOjac_constr,label='Wjac_constr', linewidth=2)


plt.legend(loc='upper left')
plt.grid()                

I am simultaneously trying to try lmfit capabilities in this post. Hence though of bringing attention to this post as well.

edit: code has been re formatted to un/comment individual method.

edit2:Jacobian is defined correctly.

edit 3: working code. incorporated changes from the comments. deleted xip, yip in the cons subroutine.

Community
  • 1
  • 1
learnerADV
  • 13
  • 4
  • The jacobian of the objective function that you provide in `minimize` is not correct. – Stelios Nov 26 '16 at 22:43
  • @Stelios Thanks ! I have modified the Jacobian correctly. With this second method is working fine. However, I am having problem in defining the constraints definition. `() missing 2 required positional arguments: 'xip' and 'yip' `. I couldnt figure out the cause. To define constraints I need `abc, xip, yip`. I am passing all these three arguments. – learnerADV Nov 27 '16 at 11:00
  • Define the constraint functions as `lambda abc: ...` instead of `lambda abc, xip, yip: ...` – Stelios Nov 27 '16 at 11:22
  • Aww.. it worked. (1) Can I know the reason ? As xip, yip are used in cons. (2) Constraints are always >= 0 according to doc for SLSQP method. This serves my purpose. Not sure how they deal with <= 0 constraints with bounds on other variables as COBYLA cannot handle bounds. – learnerADV Nov 27 '16 at 12:12
  • 1
    (1) `xip`, `yip` are global variables. The constraint functions should be defined with only argument the optimization vector variable. Do not confuse the use of `args` in `minimize`, which applies only for the objective function and not the constraints. (2) You may always express constraints of the form `f(x)<=0` as `-f(x)>=0` – Stelios Nov 27 '16 at 13:36
  • @Stelios and others. While finalizing I have got some doubt in defining constraints. I am defining currently as `2*abc[0]*xip.sum()+abc[1]`. However, is it more reasonable to mention it as `((2*abc[0]*xip+abc[1])**2).sum()` ? (by considering it in least square sense) as the output can only be scalar ? – learnerADV Nov 28 '16 at 11:45
  • The constraint function is up to you to define according to your needs. If you have multiple (scalar) constraints (as it appears in your post) you should define one function for each of them. Note that the default (`SLSQP`) method of `minimize` accepts constraint functions that return vectors, whose elements are constrained to be positive, which is a convenient way to represent multiple (similar) constraints. For example, you could consider a single constraint function returning `2*abc[0]*xip + abc[1]`. I am not sure how the jacobian should be provided in this case though. – Stelios Nov 28 '16 at 15:12

0 Answers0