Can you anybody explain how to prove the convergence of the Expectation Maximization algorithm?
For example EM for coins problems: https://math.stackexchange.com/questions/25111/how-does-expectation-maximization-work
Asked
Active
Viewed 5,162 times
1
-
This is not really programming related and is more therotical and more suited to http://maths.stackchange.com – mathematician1975 Sep 25 '12 at 09:39
2 Answers
2
EM algorithm does maximum likelihood estimation. If you look at the log likelihood, it's not true that both E and M steps always maximize it. However, if you look at the negative free energy function, both of them always maximizes it, with respect to different things though (so kind of like coordinate descent). So yes, EM algorithm always converges, even though it might converge to bad local extrema, which is a different issue.
Take a look at the classical paper www.cs.toronto.edu/~radford/ftp/emk.pdf to learn more yourself.
Yanshuai Cao
- 1,257
- 10
- 14
-
Ahh.. It is strange Knowing the convergence is not neccessary to the programming/programmer. But I understand this is more closer to the mathematics.. But I dont know How it is completely unrelated to programmers or programming ? – Sep 26 '12 at 09:56
-
I don't understand your comment. I think you meant to add this comment to the comment up there from the user "mathematician1975" – Yanshuai Cao Sep 26 '12 at 17:12
-
thank you for your answer. yes. sorry. I should have added that comment to the different place... – Sep 27 '12 at 01:35
-3
EM algorithms do not always converge. So the specifics of your problem will be important.
static_rtti
- 53,760
- 47
- 136
- 192