I am trying to find regression model parameters using Maximum Likelihood Estimation. I have assumed that the error term follows a normal distribution. My code goes like this..
def lik(parameter):
m=parameter[0]
b=parameter[1]
sigma=parameter[2]
for i in np.arange(0,len(x)):
y_exp=m * x + b
L=(len(x)/2)*np.log(2*np.pi)+len(x)/2*np.log(sigma**2)+(1/(2*sigma**2))+sum((y-y_exp)**2)
return L
lik_model=minimize(lik,np.array([5,5,5]),method='Nelder-Mead')
print(lik_model)
I am supplying x and y from a pandas Dataframe. x contains the following data: 1,3,2,1,3.y contains the following data: 14,24,18,17,27.
I am getting the following output..
final_simplex: (array([[ 4.99995861, 10.00010678, -0.44724658],
[ 4.9999851 , 10.00003754, -0.44716575],
[ 4.9999644 , 10.0001883 , -0.4471848 ],
[ 4.99988178, 10.00019657, -0.4471765 ]]), array([17.07109792, 17.07109794, 17.07109798, 17.07109798]))
fun: 17.071097921871136
message: 'Optimization terminated successfully.'
nfev: 156
nit: 85
status: 0
success: True
x: array([ 4.99995861, 10.00010678, -0.44724658])
In x:array([4.9999,10.000106,-0.4472]), the first and the second value is correct which I verified by OLS. However, the last value which gives the standard deviation of the error term is incorrect. The correct standard deviation of error was calculated as follows:
tbl=pd.read_excel('C:/Users/Hrithik-PC/.spyder-py3/anova1.xlsx')
import statsmodels.api as sm
x=tbl['TV Ads']
y=tbl['Car Sold']
x2=sm.add_constant(x)
model=sm.OLS(y,x2)
fit=model.fit()
#print(fit.summary())
e=fit.resid
stde=np.std(e)
'stde' gives the correct standard deviation of the error term.
stde
Out[41]: 1.6733200530681507
So why am I getting an incorrect value for standard deviation of error term when I use minimize function ? I tried the same code with a different dataset. I got the correct first and second value in x:([]) after minimizing but the third value was incorrect. Rather the third value was the same as that for the previous dataset i.e. '-0.4472' Why is this happening ? Please help.. Thanks in advance
