Fastest possible OpenCV 2 OpenGL context

Question

I've been searching through the net for a few day looking for the fastest possible way to take a OpenCV webcam capture and display it on an OpenGL context. So far this seems to work OK until I need to zoom.

void Camera::DrawIplImage1(IplImage *image, int x, int y, GLfloat xZoom, GLfloat yZoom)
{
    GLenum format;
        switch(image->nChannels) {
            case 1:
                format = GL_LUMINANCE;
                break;
            case 2:
                format = GL_LUMINANCE_ALPHA;
                break;
            case 3:
                format = GL_BGR;
                break;
            default:
                return;
        }

    yZoom =- yZoom;
        glRasterPos2i(x, y);
        glPixelZoom(xZoom, yZoom);  //Slow when not (1.0f, 1.0f);
        glDrawPixels(image->width, image->height, format, GL_UNSIGNED_BYTE, image->imageData);
}

I've heard that maybe taking the FBO approach would be even faster. Any ideas out there on the fastest possible way to get an OpenCV webcam capture to an OpenGL context. I will test everything I see and post results.

`glDrawPixels()` definitely isn't a great way to do it. Something like [this](http://www.songho.ca/opengl/gl_pbo.html#unpack) might be better. — Flexo, Apr 25 '12 at 21:56
I've tried 4 different ways today. Textures don't work well because you need a power of 2 ratio and sub textures slow it down. Using glu to mipmap or any kinda mipmap slows it down. Using glImage API is slow because only mipmap works. So far, the fastest example i've tried is the one above. — aquawicket, Apr 26 '12 at 10:58
How about uploading the data to a NPOT texture with `glTexImage2D()`, and then [drawing through `glBegin(GL_QUADS)`](http://stackoverflow.com/a/4854802/176769) ? — karlphillip, May 10 '12 at 18:38

score 4 · Accepted Answer · answered May 10 '12 at 18:49

Are you sure your openGL implementation needs ^2 textures? Even very poor PC implementations (yes Intel) can manage arbitrary sizes now.

Then the quickest is probably to use a openGL Pixel buffer Sorry the code is from Qt, so the function names are slightly different but the sequence is the same

Allocate the opengl Texture

glEnable(GL_TEXTURE_2D);
glGenTextures(1,&texture);

glBindTexture(GL_TEXTURE_2D,texture);       
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);          
glTexImage2D(GL_TEXTURE_2D, 0, glFormat, width, height, 0, glFormatExt, glType, NULL ); 

glDisable(GL_TEXTURE_2D);

Now get a pointer to the texture to use the memeory

glbuffer.bind();
unsigned char *dest = (unsigned char*)glbuffer.map(QGLBuffer::ReadWrite);
// creates an openCV image but the pixel data is stored in an opengl buffer
cv::Mat opencvImage(rows,cols,CV_TYPE,dest);  
.... do stuff ....
glbuffer.unmap(); // pointer is no longer valid - so neither is openCV image

Then to draw it - this should be essentially instant because the data was copied to the GPU in the mapped calls above

glBindTexture(GL_TEXTURE_2D,texture);                       
glTexSubImage2D(GL_TEXTURE_2D, 0, 0,0, width, height, glFormatExt, glType, 0);                                                                                  
glbuffer.release();

By using different types for glFormat and glFormatExt you can have the graphics card automatically convert between opencVs BGR and typical RGBA display formats for you in hardware.

Could you please elaborate more on that? I don't understand what buffer are you binding in "glbuffer.bind()". Are you referring to an FBO? I cannot use any bind() method in an FBO... I am very interested in your answer, I have asked something similar here http://gamedev.stackexchange.com/questions/30606/opengl-fbo-to-opencv-image — Dan, Jun 14 '12 at 15:34
It's a QGLBuffer http://qt-project.org/doc/qt-4.8/qglbuffer.html a wrapper around a PBO or VBO. So glbuffer.bind is equivalent to just a glBindBufferARB, see http://www.songho.ca/opengl/gl_pbo.html — Martin Beckett, Jun 14 '12 at 15:37

Fastest possible OpenCV 2 OpenGL context

1 Answers1