Azure Kinect Body Tracking Function coordinates in Unity

Question

I am a complete newcomer to programming with C#, Unity and the Azure Kinect SDK.

As part of a project, I have the goal of reading out certain points (so called joints) of the body, e.g. neck, shoulder_left, eye_right, … . Therefore, I am working with the Azure Kinect Camera from Microsoft with the Body Tracking SDK (https://learn.microsoft.com/de-de/azure/Kinect-dk/body-sdk-setup). As part of my project I need to include the Body Tracking Function into an unity project. In connection with the camera this works very well and I can see the results visually. However, I am interested in reading out the coordinates of the tracked individual joints because I need to work with them in my next step. Hence my question:

Does anybody of you have any experience on working with the Body Tracking Function in the unity surrounding and can help on getting the coordinates of the specific joints e.g. in an extra local database?

Answer 1

The skeletal data can be accessed per frame and has 4 properties:

1. Type - The name of the joint (e.g. "Head");
2. Tracking state - The tracking confidence;
3. Position - The XYZ-coordinates of the joint;
4. Orientation - The orientation of the joint in relation to it's parent joint;

Each Joint can be accessed in the following way:

List bodies = frame.BodyFrameSource.Bodies; // Gets all the skeleton data from a specific frame...

foreach (Body body in bodies) // Itterates through every body object in the frame...
{
    Joint head = body.Joints[JointType.Head]; // Gets the joint data for specifically the head joint...
    Vector3 position = head.Position; // Gets the position data of the head joint...
}

Source:

1. https://pterneas.com/2020/03/19/azure-kinect/;
2. https://learn.microsoft.com/en-us/previous-versions/windows/kinect/dn758665(v=ieb.10);
3. https://books.google.nl/books?id=Q7UwDwAAQBAJ&pg=PA105&lpg=PA105&dq=bodyframesource&source=bl&ots=5VEGYtRy3t&sig=ACfU3U1eVs28KsG6Xcegoq3X8eGK0Qxtag&hl=nl&sa=X&ved=2ahUKEwjpwcnW06rpAhUH3aQKHVjNCuwQ6AEwBHoECAoQAQ#v=onepage&q=bodyframesource&f=false;
4. https://kinect.github.io/tutorial/lab06/index.html;
5. https://learn.microsoft.com/en-us/azure/kinect-dk/body-joints;