1

I am using the new Kinect v2 and I am getting the depth map of the Kinect. After I get the depth map I convert the depth data from Depth Space to Camera Space. As far as I understand this is done, by converting all the X,Y coordinate of each pixel to Camera Space + adding the depth value as Z coordinate (also Kinect gives the depth value in millimetres so it is also converted to hold meters). Because of this, the point cloud is actually on 2D grid extended with the depth value. The visualization also confirms this, since it is easy to notice that the points are ordered in a grid due to the above conversation.

For visualization I am using OpenGL the old fashion way (glBegin(...) and glEnd()). I want to create a mesh out of the points. I kind of managed to do it with GL_TRIANGLES, but then I have lot of duplicated vertices and edges. So I thought I should create a better triangulation with GL_TRIANGLE_STRIP, but I am stuck here because I can't come up with a good algorithm which can go through my 2D grid in a way that I can feed it to the GL_TRIANGLE_STRIP so it creates a nice surface.

The problems:

  • For each triangle's vertices I am checking the Z coordinate. If it exceeds a certain threshold I disregard the triangle => this might create holes in my 2D grid.

  • Some depth values are NaN, because the Kinect can't "see" there nothing (for example an object is too far or too close) => this also creates holes in the 2D grid.

Anybody has any suggestion what would be the best method to solve this issue?

Silex
  • 2,583
  • 3
  • 35
  • 59
  • 1
    First dump all vertices that don't add anything relevant (i.e. they barely differ from one another in depth), then make the mesh. A quick google on point cloud to optimized mesh returns http://meshlabstuff.blogspot.co.uk/2009/09/meshing-point-clouds.html. Just know that when it comes to this kind of stuff, there'll rarely be an implementation just floating around for your language of choice and instead you'll have to read whitepapers and other language solutions and build up your own implementation from there. Here's an interesting looking paper http://cdn.intechopen.com/pdfs-wm/30515.pdf – Steve Lillis Nov 24 '14 at 15:45
  • Hi Steve, thanks for the comment. I will definitely read the links you provided. As a first glans they look really promising! I thought there might be an algorithm for this kind of situation, where the point cloud actually sits on a 2D grid, but I guess as you mentioned that was only an optimistic hope! :) – Silex Nov 24 '14 at 16:48
  • Well you'll still want the output to be a 3d model, so the grid is actually misleading, because you'll want the vertices that make the smoothest edges rather than those that are best aligned to a grid. The white paper is specifically dealing with a 2d point cloud from the looks, because their sample is taken from a single camera. Good luck! – Steve Lillis Nov 24 '14 at 19:49
  • why don't you use Kinect Fusion? it will even remove the noise of the depth image, and give a smooth, colored mesh. – Clones1201 Nov 26 '14 at 01:21
  • I tried Kinect Fusion...for Kinect one it really gives a relatively nice smooth mesh, but the new Kinect v2 fusion example generates a really noisy result...I tried 3 different Kinect v2 and all of them were way more noisier, then the Kinect 1 result. Since I have to work with Kinect v2 I thought I will make my own reconstruction code and actually I have it, just the meshing is not too efficient. – Silex Nov 27 '14 at 05:01
  • @Silex that's weird, Kinect v2 have a higher fidelity in depth data, that should result in a more smooth mesh. KinectFusion's mesh extraction is also not very fast, it uses ray-casting to generate a renderable point cloud to display. – Clones1201 Nov 28 '14 at 16:01
  • @Clones1201, yes I know...I was thinking to open a thread about this issue and support it with some result images, so we can see what the professionals have to say about it. Just a side note, actually from one view Kinect v2 does a better job and the result mesh is more detailed then Kinect 1's, but if I want to do a multiple view scan (like scanning an object real time from different views), then Kinect 1 produces a better, less noisy mesh... I will open a thread about this at the beginning of next week, because I am curious what is behind this! – Silex Nov 28 '14 at 19:12

2 Answers2

1

You can try also a delanauy triangulation in 3d and look for the tetrahedons on the exterior. An easy algorithm is the bowyer-watson with tetrahedons and circumspheres. Cgal is a good example.

Micromega
  • 12,486
  • 7
  • 35
  • 72
1

If you're able to use the point cloud library, you could use the class pcl::OrganizedFastMesh< PointInT >.

http://docs.pointclouds.org/trunk/classpcl_1_1_organized_fast_mesh.html

I use it to triangulate complete depth frames.

Deepfreeze
  • 1,755
  • 1
  • 13
  • 18