I encountered the same problems. I use the point cloud example as a basis for programming. In the first few seconds there is no pose and additionally if you try to change view to "front view" the application crashes (because there is no pose he could change the view and position to, I guess).
You asked your question a month ago so I am curious if you have been able to capture the camera data. I use this code for the conversion from NV12 to RGB:
int size = (int)(buffer->width * buffer->height);
for (int i = 0; i < buffer->height; i++)
{
for (int j = 0; j < buffer->width; j++)
{
float y = buffer->data[i * buffer->stride + j];
float u = buffer->data[size+2*((i / 2) * (buffer->stride / 2) + (j / 2))];
float v = buffer->data[size+2*((i / 2) * (buffer->stride / 2) + (j / 2))+1];
TangoData::GetInstance().color_buffer[3*(i*buffer->width+j)]=y;
TangoData::GetInstance().color_buffer[3*(i*buffer->width+j)+1]=u;
TangoData::GetInstance().color_buffer[3*(i*buffer->width+j)+2]=v;;
}
}
In the width x height the Y data is saved and after that comes the UV data, alternating. The conversion is done in the shader (quicker than in the on FrameAvailable method) with a known transformation (Android YUV format):
attribute vec4 vertex;
attribute vec3 color;
uniform mat4 mvp;
varying vec4 v_color;
void main() {
gl_PointSize = 7.0;
gl_Position = mvp*vertex;
float r=color.x + (1.370705f * (color.z-128.0f));
float g=color.x - (0.698001f * (color.z-128.0f)) - (0.377633f * (color.y-128.0f));
float b=color.x + (1.732446f * (color.y-128.0f));
v_color = vec4(r/255.0f,g/255.0f,b/255.0f,1.0);
This code doesn' fully work.
- If I try to just capture the Y data (luminance and therefore I should get a grey image) there are just grey dots, not making sense. Moreover, there aren't black/white points just grey points with very little contrast.
- If I add UV I just get colours in red and yellow. If I change the UV position it's all very blue. I can't detect any objects or contours.
After a few frames the camera crushes with the following exception (The first time the exception appears it says "RAW" instead of "YUV"):
E/camera-metadata﹕ /home/ubuntu/jobs/redwood_internal/RedwoodInternal/Redwood/common/player-engine/src/camera-metadata.cc:56 YUV failed to match frame 1545.014677
-The application is very slow, needing a lot of time. I used mutex, but that didn't change anything. Has anyone solved that problem or is having it right now?
I changed the configuration a little bit, because it was said that the colour camera needs to be enabled. But I couldn't find any explanation on how this is done. I hope the following code is correct, it didn't give me any error messages:
bool TangoData::SetConfig() {
[...]
if (TangoConfig_setBool(config_, "config_enable_color_camera", true) !=
TANGO_SUCCESS) {
LOGE("config_enable_color_camera Failed");
return false;
}
if (TangoConfig_setInt32(config_, "config_color_exp", 300) !=
TANGO_SUCCESS) {
LOGE("config_color_exp Failed");
return false;
}
[...]
return true;
}
Hope some of this helps or you already have good results with the camera in Leibniz release!
