5

I am currently working on a Sign Language Recognition application, where I would like to use a Hidden Markov Model as the classification stage, meaning that I will classify a gesture/posture to obtain the relevant letter or word.

I have currently completed the first stage where I am detecting the hand. Currently I can obtain a number of parameters (features) which I can use for my machine learning stage such as:

  • convex hull of hand
  • convexity defects
  • centroid of the hand
  • bounding rotated ellipses/rectangles (e.g. obtain any angle needed in terms of rotation)
  • contour of the hand
  • moments (I am not sure what these are extactly)

These are all possible to do through openCv.

My question: once I have all these features, how can I execute the 'Feature Extraction' stage? i.e. if a machine learning algorithm, in this case the HMM requires a set of probabilities, how can I use the above information?

One idea I have is to create a special data structure with such information which uniquely identifies each gesture, but how do I feed it to the machine learning technique? (in this case the Hidden Markov Model)

Can any one be able to guide me as to what I should at least search for at this particular stage or guide me to show what is actually the real difficulty I have?

test
  • 2,538
  • 4
  • 35
  • 52

1 Answers1

5

Once you have your set of observations ready, you could feed it to the Viterbi Algorithm to detect the best state sequence that may have produced these observations. Also, you can train your HMM over a data set of samples using the Baum-Welch algorithm. You could have a look at my blog post which is a simple explanation of recognizing dynamic hand gestures using HMM (although I am NOT using openCV or scanning the contour of the hand). Hope this can help you in getting a general idea about the processing and learning phase.

Darth Coder
  • 1,738
  • 7
  • 33
  • 56
  • Thank you for your answer.. I checked out your site a lot of times and I found it quite useful. I was using JAHMM to work with Hmms, it is an interesting small package. I trained them easily using the BW algo and then I used to Forward Backward to check the probability of a sequence of observations over my current HMMs. whenever I tried to use the Viterbi I always ended up with '0' in my results and I couldn't figure out what the problem was! – test Apr 22 '13 at 13:13
  • 1
    Hey if you found my answer useful, can I ask you to Vote it up :). And as for your Viterbi problem: are you sure you trained your set right and that the implementation of Viterbi your working with is correct? There are a number of sample codes out there that can help you. Since your getting a 0, it means that your Model "thinks" that the particular observation sequence supplied to it s input can NOT be produced using the current value of HMM parameters. Look into the learning phase once again just to be sure. – Darth Coder Apr 22 '13 at 17:43