0

I'm trying to simply get a 3D CameraSpacePoint position for a given 2D ColorSpacePoint from the RGB stream.

Looking at the CoordinatorMapper Methods, the only methods for mapping from color coordinates are:

MapColorFrameToDepthSpace Maps a frame from color space to depth space. MapColorFrameToDepthSpaceUsingIBuffer Maps a frame from color space to depth space. MapDepthFrameToCameraSpace Maps a frame from depth space to camera space. MapDepthFrameToCameraSpaceUsingIBuffer Maps a frame from depth space to camera space.

The issue is that on my older PC when I try to use MapColorFrameToDepthSpace speed drops from ~33fps to ~10fps. I'm guessing it takes a while to convert 1920x1080 points from 2D to 3D, but I wish there's a faster way since I need to convert a single point for my application. Even the SDK samples (both c++ and c#) run at ~1fps for the Color to Camera conversion demos.

Even if I use MapDepthFrameToCameraSpace then MapDepthPointToCameraSpace that still converts the whole depth frame to camera space when I only need one point.

Is there a way to convert a single ColorSpacePoint to a CameraPoint ? If so how ? Otherwise how could I speed-up ColorSpace to CameraSpace mapping ?

Are there any other SDKs (libfreenect2, etc.) that offer a more efficient method of retrieving the depth for a position on the color stream ?

George Profenza
  • 50,687
  • 19
  • 144
  • 218
  • https://www.polygon.com/2017/10/25/16543192/kinect-discontinued-microsoft-announcement – Hans Passant Mar 07 '18 at 22:00
  • @HansPassant dearly noted, however that doesn't mean the sensor can't still be a useful tool – George Profenza Mar 07 '18 at 22:07
  • We wouldn't want to mislead other programmers that this is still viable, now would we? Notice posted. – Hans Passant Mar 07 '18 at 22:11
  • True. Still I hope there is a good chunk of media and installation artists, computer vision and robotics researchers, etc. that used the Kinect and might still use it and find it useful (even without support from Microsoft). I'm also interested if there's other options outside the Microsoft Kinect SDK – George Profenza Mar 08 '18 at 00:04

1 Answers1

0

I have used open source library PCL for use with RGDB cameras (intel realsence though). It might be worth checking out. I'm not sure quite how well it would work with the limitations of your laptop though.

nikosm
  • 106
  • 1
  • Thanks for the tip. Have you tested retrieving a depth point based on a color stream position using kinect 2 with PCL ? – George Profenza Mar 13 '18 at 10:29
  • Personally I haven't done it but I think it would be worth a shot if you are looking for an alternative. It works pretty well with the rgdb cameras I've used and I've seen some tutorials online on how to get it working with a kinect so it shouldn't be too bad. – nikosm Mar 13 '18 at 20:01