0

While trying to make a program for hidden markov models, I did the simplest assumption for the initial HMM of the Baum-Welch algorithm : put everything as a uniform distribution. That is,

A[i][j] = 1/statenumber;
B[i][j] = 1/observationnumber;
P[i]    = 1/statenumber;

up to a logarithm to avoid underflowing. It has the benefit of not requiring to check for normalization.

But so far, I've run into the algorithm not actually doing much. The emission matrix changes at the first iteration, but not after that, and the transition matrix and initialization vector do not evolve at all. It seems to be that the gamma matrix does not change at all.

At first I thought it was my algorithm not working out too well, but after trying it on some other HMM libraries, I seem get the same type of results.

Is it impossible to converge to the correct HMM using such an initialization, and what is the ideal method to initialize those arrays?

emesday
  • 6,078
  • 3
  • 29
  • 46
Slereah
  • 257
  • 2
  • 13
  • Do you mean `A`, `B`, and `P` are `transition probability`, `observation probability` and `initial probability` respectively? There's no problem I think. Is the data correct to train HMM? If you give the data, we can give a better answer. – emesday Apr 26 '14 at 07:09

1 Answers1

1

The Baum Welch algorithm won't work with a uniform initial distribution -- the updates will be degenerate. Try to randomize it instead.

user3311160
  • 11
  • 1