What is the use of Skeleton, Gesture period in Kinect application?

Question

I am writing a Kinect-based application. I need to implement a wave-right gesture. I found a code snippet from the Internet using C#, but I have some doubts about this code. Can you help me?

The code:

public class KinectClass
{
    public void KinectInitialize ()
    {
        m_myKinect.Start();
        m_myKinect.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated;
        m_myKinect.SkeletonFrameReady += m_myKinect_SkeletonFrameReady;
    }

    void m_myKinect_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
    {
        using (SkeletonFrame frame = e.OpenSkeletonFrame())
        {
            if (frame != null)
            {
                if (0 == frame.SkeletonArrayLength) return;
                m_skeletons = new Skeleton[frame.SkeletonArrayLength];
                if (m_skeletons == null) return;
                frame.CopySkeletonDataTo(m_skeletons);
            }
        }

        foreach (Skeleton skeleton in m_skeletons)
        {
            if (skeleton.TrackingState == SkeletonTrackingState.Tracked)
            {
                UpdateSkeletonHistory(skeleton);
                value = gestureobject.Detectgesture();
            }
        }
    }
}

In addition, there is a Gesture class which is checking wave functions:

class Gesture
{
    private List<Skeleton> SkeletonHistory;
    private List<Skeleton> localSkeletonHistory;

    public void UpdateSkeletonHistory(Skeleton SkeletonData)
    {
        SkeletonHistory.Add(SkeletonData);

        if (SkeletonHistory.Count > 90)
        {
            SkeletonHistory.RemoveAt(0);
        }

        return ;
    }

    public short Detectgesture()
    {
        short sReturnvalue = -1;
        int gesturePeriod = 2;
        int indexesPerSecond = 30;
        int indexesToCheck = gesturePeriod * indexesPerSecond;
        localSkeletonHistory = new List<Skeleton>(SkeletonHistory);

        if (m_SkeletonHistory.Count > indexesToCheck)
        {
            localSkeletonHistory = new List<Skeleton>(m_SkeletonHistory);

            if( waveright() )
            {
                Console.Writeline("waveright");
            }
        }
    }

    private bool waveright()
    {
        Console.WriteLine("wave right START");
        bool Movedright = false,
        Movedleft = false;

        float refDistance = 0.2F;

        float startPos = localSkeletonHistory[0].Joints[JointType.HandLeft].Position.X;

        for (int i = 0; i < localSkeletonHistory.Count; i++)
        {
            if (!(localSkeletonHistory[i].Joints[JointType.HandLeft].Position.Y <= localSkeletonHistory[i].Joints[JointType.Head].Position.Y))
            {
                break;
            }
            if (localSkeletonHistory[i].Joints[JointType.HandLeft].Position.X >= (startPos + refDistance))
            {
                Movedright = true;
            }
            if (Movedright && Math.Abs(localSkeletonHistory[i].Joints[JointType.HandLeft].Position.X - startPos) <= 0.1 )
            {
                Movedleft = true;
                m_SkeletonHistory.Clear();
                break;
            }
        }

        return Movedleft;
    }
}

I can understand Kinect functions, but I am confused about the gesture functions. My questions are as follows:

• Why do they create a SkeletonHistory?
• What is the need for gesturePeriod and indexesPerSecond?
• Why do they check if (m_SkeletonHistory.Count > indexesToCheck)?
• Can you explain the waveright function?

Answer 1

The thing you need to remember is that just because the Hand's position is right, relative to the body. It does not mean that they are waving right.

A wave right is a transition towards the right. So you cannot determine that by the current position. That is why you need to keep track of the previous positions using SkeletonHistory.

So a blunt overview of the algorithm is:

For the previous positions since the last triggered gesture:
     If each Skeleton's Y position is greater than the last, then the person is waving right

Of course, a hand will move right many times, it does not mean it is a wave right. So you need to determine how long a hand must be moving right, for it to be a genuine waveRight gesture.

refDistance is a way of saying that the hand needed to move at least this distance before it can be categorised as a wave right.

Here, I believe the coder is saying that the gesture needs to last at least 2 seconds. So if we get 30 skeletons per second, then we need to check the last 60 skeletons:

int indexesToCheck = gesturePeriod * indexesPerSecond;

The msdn documentation is actually very good for this stuff:

https://msdn.microsoft.com/en-us/library/jj131025.aspx

I found this book excellent for getting started on this stuff:

Programming with the Kinect for Windows Software Development Kit (Developer Reference)