Kinect SDK 1.5 - Face Tracking : WPF tracking problems

Question

I'm working with the new face tracking SDK of Kinect (Microsoft Official), and I noticed that there's difference in detection between c++ and c#-wpf example: the first one is way faster in recognition than the second (the one I want to use, actually). In the c++ version the face tracking is almost on the fly, while in the wpf one it starts ONLY when I put my entire body (so the entire skeleton) in the FOV of Kinect.

Did anyone found out why? I noticed that the skeletonframe provided shows the property "Trackingmode = default", even though I set the kinect skeleton stream on seated.

colorImageFrame.CopyPixelDataTo(this.colorImage);
depthImageFrame.CopyPixelDataTo(this.depthImage);
skeletonFrame.CopySkeletonDataTo(this.skeletonData);

// Update the list of trackers and the trackers with the current frame information
foreach (Skeleton skeleton in this.skeletonData)
{
   if (skeleton.TrackingState == SkeletonTrackingState.Tracked
       || skeleton.TrackingState == SkeletonTrackingState.PositionOnly)
   {
       // We want keep a record of any skeleton, tracked or untracked.
       if (!this.trackedSkeletons.ContainsKey(skeleton.TrackingId))
       {
           this.trackedSkeletons.Add(skeleton.TrackingId, new SkeletonFaceTracker());
       }

          // Give each tracker the upated frame.
          SkeletonFaceTracker skeletonFaceTracker;
          if (this.trackedSkeletons.TryGetValue(skeleton.TrackingId, 
                                                out  skeletonFaceTracker))
          {
             skeletonFaceTracker.OnFrameReady(this.Kinect, 
                                              colorImageFormat, 
                                              colorImage, 
                                              depthImageFormat, 
                                              depthImage, 
                                              skeleton);
             skeletonFaceTracker.LastTrackedFrame = skeletonFrame.FrameNumber;
          }
    }
}

The code is the one provide my microsoft with the 1.5 SDK.

@OutlawLemur: Code added, but it's the one provided by Microsoft. The question was about the difference in detection between examples, why is the c++ tracking even when I'm seated, while in the WPF I have to stand up? Am I the only one to experience this kind of problem? — MDaldoss, May 24 '12 at 16:15
See [SkeletonTrackingMode](http://msdn.microsoft.com/en-us/library/microsoft.kinect.skeletontrackingmode.aspx) for details on the tracking modes, specifically `Seated` — Liam McInroy, May 25 '12 at 00:23
@OutlawLemur Thanks for the help but the difference was in the procedures called in the example (different approaches). Still, I can't understand why, even if the sensor is set as 'seated' the skeletonframe shows 'default' as the tracking method. — MDaldoss, May 25 '12 at 14:43

score 1 · Accepted Answer · answered May 25 '12 at 14:37

I had some information in other forums, specifically here (Thanks to this guy (blog)):

MSDN forum link

Basically, in the c++ example all the methods to track the face are used, both color+depth and color+depth+skeleton, while in the c# only the latter is used. So it only starts when you stand up.

I did some tests, but the other method is still not working for me, I did some modification to the code but with no luck. Here is my modification:

internal void OnFrameReady(KinectSensor kinectSensor, ColorImageFormat colorImageFormat, byte[] colorImage, DepthImageFormat depthImageFormat, short[] depthImage)
        {
            if (this.faceTracker == null)
            {
                try
                {
                    this.faceTracker = new Microsoft.Kinect.Toolkit.FaceTracking.FaceTracker(kinectSensor);
                }
                catch (InvalidOperationException)
                {
                    // During some shutdown scenarios the FaceTracker
                    // is unable to be instantiated.  Catch that exception
                    // and don't track a face.
                    //Debug.WriteLine("AllFramesReady - creating a new FaceTracker threw an InvalidOperationException");
                    this.faceTracker = null;
                }
            }

            if (this.faceTracker != null)
            {
                FaceTrackFrame frame = this.faceTracker.Track(
                    colorImageFormat,
                    colorImage,
                    depthImageFormat,
                    depthImage,
                    Microsoft.Kinect.Toolkit.FaceTracking.Rect.Empty);
                    //new Microsoft.Kinect.Toolkit.FaceTracking.Rect(100,100,500,400)); 

                this.lastFaceTrackSucceeded = frame.TrackSuccessful;
                if (this.lastFaceTrackSucceeded)
                {
                    if (faceTriangles == null)
                    {
                        // only need to get this once.  It doesn't change.
                        faceTriangles = frame.GetTriangles();
                    }

                    this.facePointsProjected = frame.GetProjected3DShape();

                    this.rotationVector = frame.Rotation;
                    this.translationVector = frame.Translation;
                    this.faceRect = frame.FaceRect;
                    this.facepoints3D = frame.Get3DShape();
                }
            }
        }

frame.TrackSuccessful is always false. Any idea?

score 1 · Answer 2 · answered Dec 18 '12 at 06:33

1

I finally figured it out and made a post on MSDN forums regarding what else needs to be done to get this working.

It's here.

Hope that helps!

answered Dec 18 '12 at 06:33

user1902847

41
2

Kinect SDK 1.5 - Face Tracking : WPF tracking problems

2 Answers2

Linked