0

I'm using multiple Azure Kinect devices to create a merged PointCloud with PCL and Open3D libraries. This is because Azure Kinect doesn't support multi-device body tracking fusion. I've read some people computing joints (position and orientation) from every single Kinect and then fusing them in different ways, such as Kalman filter, but the most correct way to obtain a good tracking is using a merged Cloud and then track detected bodies, but I can't find any project or SDK to use, just scientific researches. Can anyone help me? Thank you very much.

Alessandro Minerba
  • 1
  • 1
  • Did you take a look at MoveBox: https://github.com/microsoft/MoveBox-for-Microsoft-Rocketbox ? – asergaz May 19 '22 at 13:53
  • Thanks for your answer, but as the project's README says "We created a Motion Capture (MoCap) studio using a single depth sensor, such as Azure Kinect or Windows Kinect V2." they seems to use a single depth sensor, while I need an online MotionaCapture / BodyTracking tool for multiple Azure Kinect camera, in particular for their merged pointcloud in PCL or PLY formats as an input – Alessandro Minerba May 23 '22 at 07:49

1 Answers1

0

I think the reason that you're unable to find any sort of library to use because none exist! If you're able to fuse the pointclouds successfully you could try running the body tracking on that to see if it improves results, or turn that into a mesh and use some sort of mesh-based pose estimation.

Breadman10
  • 81
  • 6
  • The whole point is we can't find any library that can use a point cloud as an input and give a body tracking as an output.. – Alessandro Minerba May 24 '22 at 14:17
  • Well that is what the kinect body tracking does but I see your point. My experience is limited, but I don't know of any other ways to do it, even after looking through paperswithcode. Does the quality of the skeleton from the kinect body tracking improve when you have the fused pointcloud? – Breadman10 May 24 '22 at 18:36
  • 1
    There's no way to use the point cloud with Azure Kinect SDK because it need a Kinect.Image object as input, not a point cloud and there's no way to transform between them because the Image object only represents a 2D scene, while the pointcloud is a 3D scene – Alessandro Minerba May 24 '22 at 22:22
  • Ahhh I see, I apologize for misunderstanding. That is a tough problem... I'm afraid I don't have a good answer for you – Breadman10 May 25 '22 at 13:13