Learning a single parameter in pymc

Question

I am a beginner with pymc, and I am also quite new to Bayesian learning. Thus, this question might seem awkward due to a lack of understanding.

I worked through the tutorial, and afterwards I tried my own example. I generated some exponentially distributed data x using x = np.random.exponential(0.25, 500), and then I wanted to learn the lambda parameter that specifies the distribution (which is 1/0.25=4 in this case).

from pymc3 import Model, Exponential, find_MAP

# x is an array of pre-generated exponentially distributed data points
basic_model = Model() 
with basic_model:
    Y_obs = Exponential("Y_obs", lam=4, observed=x)

map_estimate = find_MAP(model=basic_model)

However, I get:

ValueError: zero-size array to reduction operation maximum which has no identity

I later managed to get results that seem right using this code:

from pymc3 import Model, Exponential, HalfNormal, find_MAP, sample, traceplot
basic_model = Model()

with basic_model:
    lam = HalfNormal("lam", sd=1)
    Y_obs = Exponential("Y_obs", lam=lam, observed=x)
    start = find_MAP(model=basic_model)
    trace = sample(2000, start=start)

traceplot(trace)

Is that right? What I don't understand is why I need another distribution as the lambda parameter (is HalfNormal even feasible?) to model the exponential distribution, i.e. why it is insufficient to just specify one distribution, namely the exponential distribution I am interested in.

I am not sure whether my lack of understanding lies in pymc or in the statistical problem in general.

Thank you for clarification!

score 1 · Accepted Answer · answered May 20 '16 at 22:06

In Bayesian statistics if you don't know the value of a parameter then you have to specify a prior for it.

When you write

Y_obs = Exponential("Y_obs", lam=4, observed=x)

You are saying that you know the value of your parameter with zero uncertainty, is exactly 4.

Instead in the second example you are doing the right thing. You are specifying the prior, a HalfNormal distribution and, then the likelihood (the exponential). If you check Bayes' theorem you will notice that you have two terms a prior and a likelihood. You always need at least 1 prior and 1 likelihood and in general for more complex models you will have more than one prior (one for each parameter you want to learn)

Learning a single parameter in pymc

1 Answers1