Posterior predictive distribution

In Bayesian statistics, the posterior predictive distribution is the distribution of possible unobserved values conditional on the observed values.

Given a set of N i.i.d. observations ${\displaystyle \mathbf {X} =\{x_{1},\dots ,x_{N}\}}$, a new value ${\displaystyle {\tilde {x}}}$ will be drawn from a distribution that depends on a parameter ${\displaystyle \theta \in \Theta }$, where ${\displaystyle \Theta }$ is the parameter space.

${\displaystyle p({\tilde {x}}|\theta )}$

It may seem tempting to plug in a single best estimate ${\displaystyle {\hat {\theta }}}$ for ${\displaystyle \theta }$, but this ignores uncertainty about ${\displaystyle \theta }$, and because a source of uncertainty is ignored, the predictive distribution will be too narrow. Put another way, predictions of extreme values of ${\displaystyle {\tilde {x}}}$ will have a lower probability than if the uncertainty in the parameters as given by their posterior distribution is accounted for.

A posterior predictive distribution accounts for uncertainty about ${\displaystyle \theta }$. The posterior distribution of possible ${\displaystyle \theta }$ values depends on ${\displaystyle \mathbf {X} }$:

${\displaystyle p(\theta |\mathbf {X} )}$

And the posterior predictive distribution of ${\displaystyle {\tilde {x}}}$ given ${\displaystyle \mathbf {X} }$ is calculated by marginalizing the distribution of ${\displaystyle {\tilde {x}}}$ given ${\displaystyle \theta }$ over the posterior distribution of ${\displaystyle \theta }$ given ${\displaystyle \mathbf {X} }$:

${\displaystyle p({\tilde {x}}|\mathbf {X} )=\int _{\Theta }p({\tilde {x}}|\theta )\,p(\theta |\mathbf {X} )\operatorname {d} \!\theta }$

Because it accounts for uncertainty about ${\displaystyle \theta }$, the posterior predictive distribution will in general be wider than a predictive distribution which plugs in a single best estimate for ${\displaystyle \theta }$.