I am trying to do some basic level robotics with computer vision, and I have run into a problem with going from real world coordinates to pixels in order to read from the Kinect's depth map. I am doing this because I would like to be able to draw on a hand of non constant depth, which is located at a given x and y location in millimeters.
The problem I'm having is that the distance of the object from the camera needs to be known in order to convert between real world and pixels. How could I either find the depth at a location, or convert the depth map into metric coordinates instead of pixels?
