0

What is the relationship between fisher information and EM algorithm? When I read papers about EM algorithm, people sometimes discussed fisher information, and there are algorithms which would combine fisher scoring method and EM algorithm together. However, I couldn't find materials clearly illustrate how fisher information is related to EM algorithm and what role it is playing ?

Could anyone help me understand if there is any connection?

TylerH
  • 20,799
  • 66
  • 75
  • 101
Jack2019
  • 275
  • 2
  • 10
  • suggest you post this on mathematics or cross validated. I have added an answer below which I think is right but would benefit from the guys on mathematics or cross validated casting their eyes over it. – TooTone Jun 04 '13 at 22:20

1 Answers1

2

They are connected via the Cramer-Rao lower bound. This gives the minimum possible variance of an unbiased estimator as the reciprocal of the Fisher information.

Also the maximum likelihood estimator for theta converges in distribution to N(theta, 1/(fisher information)).

TooTone
  • 7,129
  • 5
  • 34
  • 60