2

I'm training HMMs implemented in JaHMM with sensor data from an accelerometer and a gyroscope obtained from an Android wearable.

The HMM trained with accelerometer data outputs fine learned states, and has a somewhat acceptable error rate.

Both HMM's is initialized as follows:

    Hmm<ObservationVector> hmm = new Hmm<>(2, new OpdfMultiGaussianFactory(3));

    hmm.setPi(0, 0.5);
    hmm.setPi(1, 0.5);

    hmm.setOpdf(0, new OpdfMultiGaussian(
            new double[]{0,0,0},
            new double[][] {{0.1,0,0},
                            {0,0.1,0},
                            {0,0,0.1}
            }));
    hmm.setOpdf(1, new OpdfMultiGaussian(
            new double[]{0,0,0},
            new double[][] {{0.1,0,0},
                            {0,0.1,0},
                            {0,0,0.1}
                            }));


    hmm.setAij(0, 0, 0.5);
    hmm.setAij(0, 1, 0.5);

    hmm.setAij(1, 0, 0.5);
    hmm.setAij(1, 1, 0.5);

Output for HMM trained with accelerometer data:

HMM with 2 state(s)

State 0 Pi: 0.5000000000000188 Aij: 0.5 0.5 Opdf: Multi-variate Gaussian distribution --- Mean: [ 0.036 -0.051 0.075 ]

State 1 Pi: 0.5000000000000188 Aij: 0.5 0.5 Opdf: Multi-variate Gaussian distribution --- Mean: [ 0.036 -0.051 0.075 ]

However, the HMM trained with gyroscope data can't seem to learn the states of the HMM no mater how many training iterations I've tried (500 iterations). The e.g. learned state probabilities is just NaN

Output for HMM trained with gyroscope data:

HMM with 2 state(s)

State 0 Pi: NaN Aij: ? ? Opdf: Multi-variate Gaussian distribution --- Mean: [ ? ? ? ]

State 1 Pi: NaN Aij: ? ? Opdf: Multi-variate Gaussian distribution --- Mean: [ ? ? ? ]

What could be the cause for this behavior? Is there a preprocessing or normalize step i need to perform before the data is usable in the HMM? Is the number of states in the HMM that is insufficient? I've tried with five states, but it yields the same result.

A snippet of the training file for accelerometer can be seen here: https://gist.github.com/Gudui/91d2c6b2452f1ea6a5c925b1eed9b40c

A snippet of the training file for gyroscope can be seen here: https://gist.github.com/Gudui/987cc1c1a7c0311a03988b818e7cbbcb

For both training files, each line represent a training sequence.

The library is available here: https://github.com/tanjiti/jahmm

Thanks in advance!

Peter
  • 799
  • 4
  • 8
  • 23
  • I am just guessing by looking at your data - but 2 things you should try are: (1) init the Gaussians to random values (2) normalize and whiten your data, that is make sure the data has zero mean and uncorrelated coordinates (e.g by PCA). Then try hmm again. – Itamar Katz Mar 19 '17 at 08:05
  • I am uncertain about (1) What do you mean? I will accept that as an answer, if you explain. Tried (2) and normalized my gyroscope values, which yielded acceptable results. Thank you! @ItamarKatz – Peter Mar 19 '17 at 14:03
  • See my answer about (1) – Itamar Katz Mar 19 '17 at 15:09

1 Answers1

1

Elaborating my comment, I suggest:

  1. Use random initialization of the Gaussian pdf, that is instead of initializing the means vector to [0,0,0], and the covariance matrix to 0.1 times the identity matrix, as you do now, use some random values or some empirical mean and covariance based on your data.
  2. Whiten your data, that is make sure it has zero mean and unit variance in each coordinate (or even use PCA to make the coordiantes uncorrelated.
Itamar Katz
  • 9,544
  • 5
  • 42
  • 74
  • Just a quick comment @Itamar Katz , what exactly is random values for the means vector and the covariance matrix, and how would i extract or calculate emprical means and covariance based on my data? – Peter Mar 22 '17 at 09:43
  • the learning of hmm is iterative, you provide initial values using `hmm.setOpdf` (I am guessing, I don't know this library). – Itamar Katz Mar 22 '17 at 10:29