Minimizing Two Functions Simultaneously

Question

I'm trying to optimize a model that involves two functions to be fitted to data simultaneously whilst sharing variables. I am currently having problems minimizing this model due to scipy.optimize requiring a function to be passed through but I need both to be minimized together.

Simplified block of code (carried out in symfit):

dictionary = ({y1: 3x**4 + 0.5y**3 + 2z**2 + w,
               y2: x*y * exp(z**2 - w)})
mixed_model = CallableNumericalModel(dictionary, 
                connectivity_mapping = {y1:{x,y,z,w}, 
                                       y2:{x,y,z,w}})

model_sim = mixed_model(x=xdata, y=y, z=z, w=w)
res = minimize(mixed_model.ravel(), x0=2, 
               args=(y,z,w),
               options={'eps':1e-3})
fit = Fit(res,x=xdata, y1=rhoxx_exp, y2=rhoxy_exp)    
fit_result = fit.execute()
print(fit_result)

When I run the real model with the data I get an attribute error of:

ValueError: `f0` passed has more than 1 dimension.

With the traceback indicating this error occurring in the line:

res = minimize(mixed_model.ravel(), x0=2,

As I have stated it is because I have passed a model into the minimize function when it must be a function BUT I am unsure how to minimize both y1 and y2 at the same time to then fit it.

Any suggestions would be hugely appreciated, thank you in advance!

Edited as suggested below but with the real functions:

def y1(B,n_1,n_2,n_3,n_4,mu_1,mu_2,mu_3,mu_4):
    for i in range(len(dictionary)):
        return (np.exp(B[i]/f)+np.exp(-B[i]/f))
    return ( (rho1 / (rho1**2 + (R1*B)**2) + rho2 / (rho2**2 + (R2*B)**2) 
    
    + rho3 / (rho3**2 + (R3*B)**2) + rho4 / (rho4**2 + (R4*B)**2)) 
    
    / ((rho1 / (rho1**2 + (R1*B)**2) + rho2 / (rho2**2 + (R2*B)**2) 
    
    + rho3 / (rho3**2 + (R3*B)**2) + rho4 / (rho4**2 + (R4*B)**2))**2
       
       + ((-R1*B)/(rho1**2+(R1*B)**2) + (-R2*B)/(rho2**2+(R2*B)**2)
                                                
       + (-R3*B)/(rho3**2+(R3*B)**2) + (-R4*B)/(rho4**2+(R4*B)**2))**2)
    
    + (a/(np.exp(B/f)+np.exp(-B/f))/2)*((rho1 / (rho1**2 + (R1*B)**2) + rho2 / (rho2**2 + (R2*B)**2) 
    
    + rho3 / (rho3**2 + (R3*B)**2) + rho4 / (rho4**2 + (R4*B)**2)) 
    
    / ((rho1 / (rho1**2 + (R1*B)**2) + rho2 / (rho2**2 + (R2*B)**2) 
    
    + rho3 / (rho3**2 + (R3*B)**2) + rho4 / (rho4**2 + (R4*B)**2))**2
       
       + ((-R1*B)/(rho1**2+(R1*B)**2) + (-R2*B)/(rho2**2+(R2*B)**2)
                                                
       + (-R3*B)/(rho3**2+(R3*B)**2) + (-R4*B)/(rho4**2+(R4*B)**2))**2)) )

def y2(B,n_1,n_2,n_3,n_4,mu_1,mu_2,mu_3,mu_4):
    return  ( - ((-R1*B)/(rho1**2+(R1*B)**2) + (-R2*B)/(rho2**2+(R2*B)**2)
                                                
       + (-R3*B)/(rho3**2+(R3*B)**2) + (-R4*B)/(rho4**2+(R4*B)**2)) / 
    
  ((rho1 / (rho1**2 + (R1*B)**2) + rho2 / (rho2**2 + (R2*B)**2) 
    
    + rho3 / (rho3**2 + (R3*B)**2) + rho4 / (rho4**2 + (R4*B)**2))**2
       
       + ((-R1*B)/(rho1**2+(R1*B)**2) + (-R2*B)/(rho2**2+(R2*B)**2)
                                                
       + (-R3*B)/(rho3**2+(R3*B)**2) + (-R4*B)/(rho4**2+(R4*B)**2))**2)  )
    
def f(x0):
    B,n_1,n_2,n_3,n_4,mu_1,mu_2,mu_3,mu_4 = x0
    error = []
    expected_output = [(1,2), (3,4)]
    for i in expected_output :
        error += [y1(B,n_1,n_2,n_3,n_4,mu_1,mu_2,mu_3,mu_4) - i[0],
                  y2(B,n_1,n_2,n_3,n_4,mu_1,mu_2,mu_3,mu_4) - i[1],
                 ]
    return error
x0 = (0, 1.51e27, -1.519e27, 0.915e27, -1.047e27, 1.41, 0.64, 0.087, 0.09)
result = scipy.optimize.leastsq( f, x0, epsfcn = 1e-3)

shows an error:

IndexError: invalid index to scalar variable.

found in function y1 in the line with the exponentials

Any further advice would be amazung, thank you in advance!

You are mixing symfit and scipy. Just pass `mixed_model` to `Fit` and let symfit do the rest. So like this: `Fit(mixed_model, x=xdata, y1=rhoxx_exp, y2=rhoxy_exp, w=..., z=...) `, where you have to pass all the relevant data to fit. `Fit` will string everything together behind the scenes. — tBuLi, Oct 10 '22 at 11:37

score 0 · Answer 1 · answered May 10 '21 at 21:11

You have to be tricky but it's not that difficult to trick minimize. The point is that minimize only works for scalar cost functions but leastsq can accept a tuple as an argument. So, if somehow you can work with using leastsq to minimize your function instead. The approach would go something like this.

Define your functions that you like to minimize y1(w,x,y,z), y2(w,x,y,z), ...
Define your function to be minimized as f(), where x0 is expanded to the parameter tuple.
The function f returns a vector of differences between discrete measured sample and the individual functions y1, y2 etc.
Let SciPy minimize this function, starting with a reasonably selected initial parameter vector.

For some reason I can't really get the formatting right on Stackoverflow.

So here's the code on pastebin.

https://pastebin.com/YxhkJgzD

Thank you for your answer. I have edited the question to show how I applied this and now I am getting an index error of "invalid index to scalar variable" which stems from the exponentials found in the function y1. Any advice on how to fix this please? — Ryan, May 12 '21 at 08:53

Minimizing Two Functions Simultaneously

1 Answers1