2

I am using a web cam to get video feed and then performing motion tracking on this video feed. The motion tracker returns (x,y) co-ordinates continuously.

I want to use these (x,y) to recognize gestures such as "swipe left", "swipe right", "swipe up" or "swipe down".

How do i make and store templates of these gestures and how do i figure out/recognize if one of the gestures has happened ?

Thank you in advance :)

PS: I am using Flex 4 and ActionScript 3.0. If someone could help me out with the logic, i can write it in ActionScript.

codemaniac
  • 879
  • 1
  • 11
  • 31
  • I ran into [link]http://gestureworks.com before, it may not be a ready solution to your problem, but hopefully the logic inside may inspire a solution. – Tianzhen Lin Jul 11 '12 at 05:47

3 Answers3

3

An approach I could think of working would be to have a series of (X,Y) coordinates representing points along the gesture. On a small scale if a gesture that passed through your screen was graphed as such:

|1|1|1|

|1|0|0|

|1|0|0|

and represented (from the upper left corner representing 0,0):

(0,2)(0,1)(0,0)(0,1)(0,2)

Break the x,y coordinates up into individual 2d arrays with total distance traveled between the current coordinate and the first point (in all cases in this example it would increment by 1) so you would have two arrays:

X:(0,0)(1,1)(2,2)

Y:(0,1)(1,1)(2,2)

Now do a least square fit on each array to find the closest representation of the change in x and change in y as quadratic functions. Do the same to your per-deteremined gestures and then plug in the x,y coordinates of your per-determined gestures into to the user's gesture's quadratic function and the per-determined gestures you designed and see which one it matches the closest. This is your gesture.

(I've never tried processing gestures, but I don't see why this wouldn't work)

retrohacker
  • 3,194
  • 4
  • 21
  • 31
2

You should divide your task into smaller subtasks. In computer vision there is no thing like a generic gesture detection that works out of the box in all environments.

First of all, you need to be able to detect motion at all. There are several ways to do this, e.g. background subtraction or blob tracking.

Then you need to extract certain features from your image, e.g. a hand. Again, there is more than one way to do this. Starting from skin color approximation/evaluation, which is very error prone to different lighting conditions, to more advanced techniques which really try to "analyze" the shape of an object. Those algorithms "learn" over time how a hand should look like.

I can only recommend to buy a decent book about computer vision and to research the web for articles etc. There are also libraries like OpenCV you can use for learning more about the implementation side. There should be several ports of OpenCV to ActionScript 3. I also can recommend the articles and tools from Eugene Zatepyakin (http://blog.inspirit.ru). He's doing great CV stuff with ActionScript 3.

Long story short, you should research motion tracking and feature extraction.

mschloesser
  • 71
  • 4
1

The best place to start is to read about how sign language recognition or trackpad input works, such as creating reference images and comparing them to user input. Specific to Adobe, there's the FLARToolKit, which is detailed in an augmented reality article on their website.

References:

Trackpad Science

Hand Gesture Recognition

Sign Language Recognition Research - PDF

Gesture Recognition Walkthrough - Video

Paul Sweatte
  • 24,148
  • 7
  • 127
  • 265