I am receiving multiple depth frames from the network and integrating into reconstruction using the modified Kinect Fusion C++ example:
// smoothing
col_reconst->SmoothDepthFloatFrame(m_pDepthFloatImage, m_pDepthFloatImage, 3, m_fMaxDepthThreshold);
col_reconst->SmoothDepthFloatFrame(m_pDepthFloatImage2, m_pDepthFloatImage2, 3, m_fMaxDepthThreshold);
col_reconst->SmoothDepthFloatFrame(m_pDepthFloatImage3, m_pDepthFloatImage3, 3, m_fMaxDepthThreshold);
hr = m_pVolume->IntegrateFrame(m_pDepthFloatImage, m_cMaxIntegrationWeight, &m_worldToCameraTransform);
hr2 = m_pVolume->IntegrateFrame(m_pDepthFloatImage2, m_cMaxIntegrationWeight, &m_worldToCameraTransform2);
hr3 = m_pVolume->IntegrateFrame(m_pDepthFloatImage3, m_cMaxIntegrationWeight, &m_worldToCameraTransform4);
I set the matrices m_worldToCameraTransform for each one manually, using scrollbars for rotation angles and distances.
The problem consists of the following:
- Of course, I cannot get excellent precision with manual calibration, but the things become even worse as it looks like depth frames (not the physical devices) when added into reconstruction are "shadowing" each other and discrepancies result in lower quality of combined reconstruction when compared to single device used from one viewpoint. I will try to show it on the screenshots here:
Please notice that the first and the second screenshots indicate that Kinect does not seem to take viewpoint angles (perspective) into account.
This effect can be decreased after very precise positioning, but the best results I could achieve are still poor:
- Seems to be the result of 1: I can apply Kinect Fusion reconstruction this way only to a single frame (i.e. for one moment with 3 depth frames integrated) and need to reset reconstruction constantly, as adding next data ruins it completely. This is not a big problem, as I want to do live 3d reconstruction, but is there any way to get better shape?
I was thinking about exporting mesh and using some mesh smoothing, but may be I'm just doing something wrong with API? Can I make different parts not shadow each other for example when there is minor inconsistency?
Thank you!
