0

I am currently working on a project where the Azure Kinect is meant to be used as an interaction method for a 360° Screen room.

When using the Body Tracking Viewer from Microsoft the camera recognizes me well and precisely but in my application the calculated distance (using distanceTo from each vector) between my handtip joint and thumb joint is all over the place. Without moving my hand it will go from 10mm to 120mm. I absolutely need this to be a bit more precise to be able to select things with the hand.

I suppose I have a problem in my startup configuration in the camera which causes it to be less precise than the Body Tracking Viewer but I don't know where to look.

As I wasn't sure if the default tracker config uses the lite dnn_model_2_0 or the "big" one, I tried setting it manually tracker_config.model_path = "dnn_model_2_0_op11.onnx"; but with the same result. Going to near field of view instead of wide does help a little bit but it stays very janky. Especially the left hand is all over the place, the right one is almost precise.

Here the startup/config from my cpp project

[...]
MQTTClient client;
MQTTClient_connectOptions conn_opts = MQTTClient_connectOptions_initializer;
MQTTClient_message pubmsg = MQTTClient_message_initializer;

int maint()
{
k4a_device_t device = NULL;
    VERIFY(k4a_device_open(0, &device), "\nOpen K4A Device failed! Is the camera connected?");


    // Start camera. Make sure depth camera is enabled.
    k4a_device_configuration_t deviceConfig = K4A_DEVICE_CONFIG_INIT_DISABLE_ALL;
    deviceConfig.depth_mode = K4A_DEPTH_MODE_WFOV_UNBINNED;
    deviceConfig.color_resolution = K4A_COLOR_RESOLUTION_OFF;
    deviceConfig.camera_fps = K4A_FRAMES_PER_SECOND_15;
    VERIFY(k4a_device_start_cameras(device, &deviceConfig), "\nStart K4A cameras failed!");

    k4a_calibration_t sensor_calibration;
    VERIFY(k4a_device_get_calibration(device, deviceConfig.depth_mode, deviceConfig.color_resolution, &sensor_calibration),
        "Get depth camera calibration failed!");

    k4abt_tracker_t tracker = NULL;
    k4abt_tracker_configuration_t tracker_config = K4ABT_TRACKER_CONFIG_DEFAULT;
    VERIFY(k4abt_tracker_create(&sensor_calibration, tracker_config, &tracker), "\nBody tracker initialization failed!");

[...]
}
flololan
  • 373
  • 1
  • 2
  • 13
  • Have a look at the following related threads in github where this issue is confirmed as a bug and enhancement to Body Tracking: https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1528 and https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1586 – asergaz May 19 '22 at 11:39
  • I have been all over their GitHub as you can see even the issues mentioned by you are a year old with no change.... – flololan May 19 '22 at 12:25
  • I have checked with Product Team that currently there are no plans to improve the hand tracking accuracy. – asergaz May 20 '22 at 12:21

0 Answers0