James–Stein estimator

The James–Stein estimator is a biased estimator of the mean, ${\displaystyle {\boldsymbol {\theta }}}$, of (possibly) correlated Gaussian distributed random variables ${\displaystyle Y=\{Y_{1},Y_{2},...,Y_{m}\}}$ with unknown means ${\displaystyle \{{\boldsymbol {\theta }}_{1},{\boldsymbol {\theta }}_{2},...,{\boldsymbol {\theta }}_{m}\}}$.

It arose sequentially in two main published papers. The earlier version of the estimator was developed in 1956, when Charles Stein reached a relatively shocking conclusion that while the then-usual estimate of the mean, the sample mean, is admissible when ${\displaystyle m\leq 2}$, it is inadmissible when ${\displaystyle m\geq 3}$. Stein proposed a possible improvement to the estimator that shrinks the sample means ${\displaystyle {{\boldsymbol {\theta }}_{i}}}$ towards a more central mean vector ${\displaystyle {\boldsymbol {\nu }}}$ (which can be chosen a priori or commonly as the "average of averages" of the sample means, given all samples share the same size). This observation is commonly referred to as Stein's example or paradox. In 1961, Willard James and Charles Stein simplified the original process.

It can be shown that the James–Stein estimator dominates the "ordinary" least squares approach, meaning the James–Stein estimator has a lower or equal mean squared error than the "ordinary" least square estimator.

Similar to the Hodges' estimator, the James-Stein estimator is superefficient and non-regular at ${\displaystyle \theta =0}$.