2

I am trying to estimate mean value, weight and covariance of a Gaussian Mixture using EM algorithm but I am getting "NaN" (Not a Number) for all the values of the vectors. Here is my code:

function [nw,ns,nu]=est(wo,so,uo,w,s,u,o,J)
% o is the data points vector (size=nx2)
% J is the number of Gaussian curve
% maximum number of Gaussians=5
% wo is the old weight vector (size=1x5) and "w" is the actual weight vector (size=1x5)
% so is the old covariance vector (size=1x5) and "s" is the actual covariance value (size=1x5)
% uo is the old mean vector (size=5x2) and "u" is the actual mean vector (size=5x2)

o=unique(o,'rows');
t=length(o);

for i=1:5
for j=1:J
    un=0;ud=0;s2n=0;
    for T=1:t
        ud=ud+probab(wo,o(T,:),uo,so,j,J);
    end
    for T=1:t
        un=un+probab(wo,o(T,:),uo,so,j,J)*(o(T,:));
    end
    for T=1:t
        s2n=s2n+probab(wo,o(T,:),uo,so,j,J)*(o(T,:)-uo(j,:))*(((o(T,:)-uo(j,:))'));
    end
    wtemp(j)=ud/t;
    utemp(j,:)=un/ud;
    stemp(j)=sqrt(s2n/(ud));
end
wo=wtemp;
uo=utemp;
so=stemp;
end
    nw=wo;
    ns=so;
    nu=uo;

The function "probab" is:

function [pro]=probab(w,o,u,s,j,J)
pro=w(j).*(1/(2*pi*s(j))).*exp((-1/(2.*(s(j).^2)))*((o(1)-u(j,1)).^2+(o(2)-u(j,2)).^2))/(gaussiandistribution(w,u,s,o(1),o(2),J));

The function "gaussiandistribution" is

function [z]=gaussiandistribution(w,u,s,X,Y,J)
z1=w(1).*(1/(2*pi*s(1))).*exp(-(((X-u(1,1)).^2)/(2.*s(1).^2)+((Y-u(1,2)).^2)/(2.*s(1).^2)));
z2=w(2).*(1/(2*pi*s(2))).*exp(-(((X-u(2,1)).^2)/(2.*s(2).^2)+((Y-u(2,2)).^2)/(2.*s(2).^2)));
z3=w(3).*(1/(2*pi*s(3))).*exp(-(((X-u(3,1)).^2)/(2.*s(3).^2)+((Y-u(3,2)).^2)/(2.*s(3).^2)));
z4=w(4).*(1/(2*pi*s(4))).*exp(-(((X-u(4,1)).^2)/(2.*s(4).^2)+((Y-u(4,2)).^2)/(2.*s(4).^2)));
z5=w(5).*(1/(2*pi*s(5))).*exp(-(((X-u(5,1)).^2)/(2.*s(5).^2)+((Y-u(5,2)).^2)/(2.*s(5).^2)));
if J==5
z=z1+z2+z3+z4+z5;
elseif J==4
z=z1+z2+z3+z4;
elseif J==3
z=z1+z2+z3;
elseif J==2
z=z1+z2;
elseif J==1
z=z1;
end
Qiu
  • 5,651
  • 10
  • 49
  • 56
Perissiane
  • 23
  • 6
  • "but it is not working" This is very vague. Does the code run? Does t give an error? If so, what is the error message? If not, where does it fail? No-one is going to go through all that code for you, you need to put in some effort of your own. – David May 01 '15 at 02:14
  • By saying "it is not working" I mean it gives me "NaN" for all vectors and for your information I am working on it for two months so I am not expecting syntax error or something and I am not expect someone to go through all the code. Unfortunately I do not have any statistics knowledge to find out the exact problem even I thoroughly read some papers on this topic. Maybe it is a wrong place to ask... – Perissiane May 01 '15 at 02:21
  • OK, go through the code and find where the first `NaN` appears, and tell us what variable and and what stage of the code. That will help you pin down the problem. – David May 01 '15 at 02:25
  • Actually the problem appears when the weight vector approaches zero (or a very small value). When the weight vector approaches zero, the output of "gaussiandistribution" function will go to zero which causes the output of "probab" function to go to zero. As you can see, probab function plays the biggest role in updating the mean value, weight and covariance vectors. I am suspecting "ud" variable approaching zero and making a division by zero which results "NaN". – Perissiane May 01 '15 at 03:00

1 Answers1

0

I have run your code (in Octave) and I don't see any NaN's. Here are the values I have:

octave:13> wo
wo =

   0.20000   0.20000   0.20000   0.20000   0.20000

octave:14> so
so =

   1   1   1   1   1

octave:15> uo
uo =

   0   0
   0   0
   0   0
   0   0
   0   0

octave:16> w
w =

   0.20000   0.20000   0.20000   0.20000   0.20000

octave:17> s
s =

   1   1   1   1   1

octave:18> u
u =

   0   0
   0   0
   0   0
   0   0
   0   0

octave:19> 
octave:19> o
o =

   1.00000   2.00000
   3.00000   1.00000
  -1.00000   2.00000
   2.00000  -2.00000
  -2.00000  -1.00000
   3.00000  -2.00000
   2.00000   3.00000
   1.50000  -0.25000

and here is the output I get:

octave:20> [nw, ns, nu] = est(wo, so, uo, w, s, u, o, 5)
nw =

   0.20000   0.20000   0.20000   0.20000   0.20000

ns =

   2.4770   2.4770   2.4770   2.4770   2.4770

nu =

   1.18750   0.34375
   1.18750   0.34375
   1.18750   0.34375
   1.18750   0.34375
   1.18750   0.34375

I didn't check whether these are correct, but anyway they are not NaN.

What values do you have for the inputs?

Robert Dodier
  • 16,905
  • 2
  • 31
  • 48
  • Here are my values: we=[0.2,0.5,0.3,0,0],ue=[12,12;10,10;9,9;0,0;0,0],se=[0.5,0.5,0.5,1,1],w=[0.3,0.3,0.4,0,0],u=[20,20;15,30;30,20;0,0;0,0],s=[3,1.6,2,1,1],J=3. And when I apply a large "o" vector it is more likely to get "NaN". – Perissiane May 01 '15 at 03:06
  • If you get NaN sometimes, sometimes not, then the value of `o` seems to be important ... so what is the value of `o`? By the way, when I run `est` with the parameters you gave, and the same `o` which I used before, I get an array index out of bounds. Not sure what's going on there; I don't see any obvious error. – Robert Dodier May 01 '15 at 03:27
  • The values of "o" should be between 0 to 1. And my "o" is quite large. have you set J as 3? – Perissiane May 01 '15 at 03:36
  • Well, please bear in mind that I can't test your code if you don't tell me exactly what values of `o` you are using. Ideally you'll tell me a value which yields NaN's and another which doesn't. If `o` is "quite large", maybe you need to look for a smaller example which shows the behavior. – Robert Dodier May 01 '15 at 16:03
  • How can I upload the variable vector? – Perissiane May 01 '15 at 16:34
  • Maybe upload the data to pastebin.com and then copy the link here. – Robert Dodier May 01 '15 at 16:36
  • pastebin.com/gLNKf8bN – Perissiane May 01 '15 at 17:15
  • Actually the problem appears when the weight vector approaches zero (or a very small value). When the weight vector approaches zero, the output of "gaussiandistribution" function will go to zero which causes the output of "probab" function to go to zero. As you can see, probab function plays the biggest role in updating the mean value, weight and covariance vectors. I am suspecting "ud" variable approaching zero and making a division by zero which results "NaN". – Perissiane May 01 '15 at 18:17
  • Thanks for posting the data. Some comments. (1) The output of gaussiandistribution is sometimes zero and that leads to division by zero and NaN. That is caused by `se` being small relative to the distance from (X, Y) to `u`. I find that if I call `est` with `100*se` instead of just `se` it makes the division by zero and NaN go away. (2) I get an array subscript out of bounds which appears to be caused by the size of `u` changing from 5 x 2 to 3 x 2 at some point. But you really only need 3 rows, right? since J = 3. Maybe you can rewrite the code to not loop over the unused rows. – Robert Dodier May 01 '15 at 18:56
  • Another comment. Remove all the semicolons from your code so that all of the calculated values are printed out. Yes, that is very verbose, but it will help you understand what's going on. – Robert Dodier May 01 '15 at 18:57
  • You are right. I have found the same problem in the literature solved by adding a number (say 1) to the numerator of the "se" formula and "1" to the denominator. so we'll have: [link](http://pastebin.com/7Fg67sRR) – Perissiane May 01 '15 at 19:45
  • This modified EM Algorithm gives me also "NaN" because the value added to the numerator is important and should be determined based on statistics methods which I am currently working on. However I have downloaded this file [link](http://www.mathworks.com/matlabcentral/fileexchange/8636-em-gm) to compare it to my file and it perfectly works with my "o" vector. It means mine is wrong. – Perissiane May 01 '15 at 19:55