0

I'm guessing this is not possible, at least with the standard HMM implementation of scikit-learn, but I wanted to ask this question to see if there are any other approaches to this issue.

The problem is basically that I want to use a hidden markov model to fit some set of observations sequences; however, the observations I have are not always aligned to the same time step. So for instance, I might have two sequences that I want to use for fitting like:

obs1 = [1,2,3,4]
obs2 = [1,2,2,4]

However, these are not necessarily evenly spaced in time. So it might be that say the observations in obs1 were observed at times 1,2,3,6, and the observations in obs2 were observed at times 1,4,5,6. Therefore it seems wrong to feed them into an HMM as is.

What I tried so far is "interpolating" the observations so that the time steps are the same. So the examples above would become:

obs1 = [1,2,3,3,3,4]
obs2 = [1,1,1,2,2,4]

This seems to work relatively well for my machine learning problem; however, it slows down the fitting considerably because of the large numbers of additional samples (the observations are relatively sparse so adding these interpolations adds hundreds of additional observations). Is there any other way to do it?

Korem
  • 11,383
  • 7
  • 55
  • 72
houbysoft
  • 32,532
  • 24
  • 103
  • 156
  • 1
    For future reference, hmms will no longer be part of scikit learn in future versions, but are to be found here https://github.com/hmmlearn/hmmlearn where they will (a priori) stay. – eickenberg Jun 26 '14 at 20:11

1 Answers1

1

I think you have hit on the simplest solution here - if it works I would continue on. A few other (possible) solutions come to mind.

  1. Instead of direct encoding, try encoding the differential i.e. 1, 1, 1, 1, 1, 2, 1 could be 1, 0, 0, 0, 0, 1, -1. This might let you "reduce down" the high rate signal to only look at states when things are actually changing. In communications this would be something like changing from BPSK (binary phase shift keying) to DBPSK (differential binary phase shift keying) - you could look into those for some ideas.

  2. Are these events actually related in a probability sense? Maybe you can actually decompose them with separate HMMs, then use each probability to feed another classifier/statistical decision. This is probably the easiest way to handle "multiple flow rates" without significant waste due to upsampling the lower signal, especially if the lower rate signal is significantly less than the higher rate.

  3. Find a feature representation to encode the low rate signal onto the high rate signal. This would allow you to combine into a single stream, but may not save any computation time.

  4. You could try randomly subsampling the higher rate signal. You would lose information, but the overall model learned might still be usable. Similar idea as compressed sensing, though it would be better to "spend" your sampling points around regions of large variation.

As @eickenberg mentioned, HMMs are moving into their own repo, dubbed HMMlearn, in the near future (0.16 I believe), so be aware!

Kyle Kastner
  • 1,008
  • 8
  • 7