Sending Kinect Body[] data over MQTT

Question

I am trying to send Microsoft's Kinect v2 body data over MQTT to effectively map skeletal data without direct connection to the Kinect, but I can't seem to deserialize a Body[] correctly. I am publishing the a list of Bodys every frame in Update().

My current setup is using Newtonsoft's JSON.Net to serialize a List taken from Body[] and publish it to MQTT (using https://github.com/vovacooper/Unity3d_MQTT). I used this since the Body class isn't serializable (so I can't use JSONUtility?).

Essentially I have:

 void Update() {
     ...
     //trackedBodies is a List<Body> that contains the tracked Bodys
     //client is MQTTClient that is connected
     string bodyData = JsonConvert.SerializeObject(trackedBodies);
     client.Publish("test", System.Text.Encoding.UTF8.GetBytes(bodyData));
     ...
 }

And:

 void client_MqttMsgPublishReceived(object sender, MqttMsgPublishEventArgs e) {
     //Check MQTT for data, then deserialize
    List<Body> bodyData = JsonConvert.DeserializeObject<List<Body>>(System.Text.Encoding.UTF8.GetString(e.Message));
     Debug.Log(bodyData);
 }

When the List is empty, I am able to receive an empty List of Bodys. When it is nonempty, the code halts for that method, and I am no longer able to receive messages at all. The Update() method still works properly.

I would appreciate it if anybody knows how to help with what I currently have, or suggest a better alternative to my problem.

score 2 · Answer 1 · answered Feb 07 '18 at 11:26

It might be too much data for converting to JSON. Maybe you should 1) look for alternative JSON libraries or 2) decode and encode the object yourself and only transmit the data you need.

Something like:

void Update() {
...
    //trackedBodies is a List<Body> that contains the tracked Bodys
    //client is MQTTClient that is connected
    string bodyData;
    foreach(Body body: trackedBodies)
        bodyData += "|" + body.X + ";" + body.Y + ";" + body.Z;

    client.Publish("test", System.Text.Encoding.UTF8.GetBytes(bodyData));
    ...
}

Also have a look here: https://www.newtonsoft.com/json/help/html/Performance.htm#ManuallySerialize

Could you please show what kind of data is inside trackedBodies?

trackedBodies is a List containing 'Body's. https://msdn.microsoft.com/en-us/library/windowspreview.kinect.body.aspx There is certainly a lot of information, glancing at the JSON. I suppose the property of main interest at this time is just Joints (property of a Body). I wanted to use JSONUtility as I've heard it was fast for Unity, but it required classes to be Serializable. Could I make a pseudo-class that contains the body information, say just the Joints? — Charlie, Feb 07 '18 at 14:40

score 0 · Accepted Answer · answered Feb 16 '18 at 01:14

I see that some people also may have the same question. I have finally gotten a working solution for myself. I personally just exposed the Body object by creating a custom 'pseudo' class to represent the body and hold the Joint and JointOrientation arrays. I was able to send a list of these custom objects through MQTT by serializing them with Json.NET.

Cheers.

Sending Kinect Body[] data over MQTT

2 Answers2