Expectation Maximization in Matlab on Missing Data

Question

I have to use EM to estimate the mean and covariance of the Gaussian distribution for each of the two classes. They have some missing attributes too.

Classes of each object is known. Therefore the problem basically reduces to fitting a gaussian model with missing element.

Which is the best library to use?

How is ECM Algorithm different from EM algorithm?

Answer 1

If you have access to the Statistics Toolbox, you can use the GMDISTRIBUTION class to fit a Gaussian Mixture Model using the EM algorithm.

Here is an example:

%# sample dataset
load fisheriris
data = meas(:,1:2);
label = species;

%# fit GMM using EM
K = 2;
obj = gmdistribution.fit(data, K);

%# assign points to mixtures: argmax_k P(M(k)|data)
P = posterior(obj, data);
[~,mIDX] = max(P,[],2);

%# GMM components
obj.mu             %# means
obj.Sigma          %# covariances
obj.PComponents    %# mixture weights

%# visualize original data clusters
figure
gscatter(data(:,1), data(:,2), label)

%# visualize mixtures found
figure
gscatter(data(:,1), data(:,2), mIDX), hold on
ezcontour(@(x,y)pdf(obj,[x y]), xlim(), ylim())

If not, check out the excellent Netlab Toolbox, as it has GMM implementation.

Answer 2

Thanks all.But I am using ecmnmle for estimating parameters and then obtaining distribution of marginals which is used later in bayes classification. It works pretty fine with accuracies of 0.9 and 0.69 on 2 classes.

Answer 3

Please take a look at the PMTK toolkit

Here is the EM implementation (Fit a mixture of Gaussians where the data may have NaN entries)