3

the output error is : MinimizerException: Cannot determine Confidence Intervals without sensible uncertainty estimates

Why I got this error? How can I calculate uncertainty estimates and solve this problem??

for dosya1 in glob.glob("mean*"):
     data1=np.genfromtxt(dosya1, skip_header=0, skip_footer=0, names=["wavelength","mean"])
     x=data1["wavelength"]

     mod=VoigtModel()
     pars = mod.guess(y, x=x)
     pars['gamma'].set(value=0.7, vary=True, expr="")     
out=mod.fit(y,pars, x=x)

     pars=lmfit.Parameters()
     pars.add_many(('amp', out.params["amplitude"].value), ('sig', out.params["sigma"].value), ("gam",out.params["gamma"].value),("cent",out.params["center"].value))

 def residual(p):
     amp=p["amp"].value
     sig=p["sig"].value
     gam=p["gam"].value
     cent=p["cent"].value  
     return  ((wofz((x-cent + wofz(gam).imag)/(sig*(sqrt(2)))).real) / (sig*(sqrt(2))))- y

 mini = lmfit.Minimizer(residual, pars)
 result=mini.minimize()

 ci = lmfit.conf_interval(mini, result)
 lmfit.printfuncs.report_ci(ci)
sirius123
  • 125
  • 1
  • 2
  • 4
  • Could you please add some more information about the format of your residual function? What is wofz? I am assuming since your minimization routine completed, you are returning a flattened array within residual. One thing you might want to try is adding reasonable bounds for the parameters you define or limiting your confidence interval calculation to a few params to start to diagnose. For example `params.add('gamma', value = -0.7, max=0, min=4)` or `lmfit.conf_interval(mini, result, p_names=['gamma']` What version of lmfit are you using? – XtremeJake May 06 '16 at 17:02

1 Answers1

4

You will get this error message if lmfit.minimize() (actually, leastsq(), which it calls) is unable to estimate uncertainties by inverting the curvature matrix. It uses these values (which are often very good estimates, BTW) as the scale for explicitly exploring parameter space. There are several possible reasons why leastsq() might fail to estimate uncertainties. Common reasons are that one or more of the variables is not found to alter the fit, or the residual contains NaNs.

It is hard to predict when this might happen. You should allow for the possibility and/or check that the initial fit succeeded and was able to make the initial estimate of the uncertainties (check result.errorbars) before calling conf_interval().

M Newville
  • 7,486
  • 2
  • 16
  • 29
  • Thanks Matt for the work on lmfit. Can you advice anything else in the case when the estimation of uncertainties fails? I.e. when there is no variable having no effect and the residuals do not contain NaNs. My impression is that leastsq quite often cannot yield the covariance matrix. Are there other ways to obtain the confidence ranges? – dnalow Sep 06 '16 at 12:10
  • well it probably depends on the details of why the uncertainty estimation failed. If all variables have a finite effect and there are no NaNs, it should work -- it should not "quite often" fail. Do you have multiple examples where it does? Also, Stackoverflow is a terrible place for such a discussion, and its use for question about lmfit should not be encouraged. Move to the appropriate mailing list. – M Newville Sep 08 '16 at 02:16
  • Having the same problem with no yielded covariance matrix. Where is the appropriate mailing list located? – Tim Oct 04 '17 at 23:28