OpenCV 2.3: Convert Mat to RGBA pixel array

Question

I am attempting to use OpenCV to grab frames from a webcam and display them in a window using SFML.

VideoCapture returns frames in OpenCV's Mat format. To display the frames, SFML requires a 1D array of pixels in its uint8 format, which (as far as I can tell) is interchangeable with uchar. This array is expected to represent 32 bits per pixel RGBA.

So, I have a uchar array, and I'm looping over the Mat data and copying each pixel:

VideoCapture cap(0);
Mat frame;
cap >> frame;

uchar* camData = new uchar[640*480*4];
uchar* pixelPtr = frame.data;
for(int i = 0; i < frame.rows; i++)
{
    for(int j = 0; j < frame.cols; j++)
    {
        camData[i*frame.cols + j + 2] = pixelPtr[i*frame.cols + j + 0]; // B
        camData[i*frame.cols + j + 1] = pixelPtr[i*frame.cols + j + 1]; // G
        camData[i*frame.cols + j + 0] = pixelPtr[i*frame.cols + j + 2]; // R
        camData[i*frame.cols + j + 3] = 255;

    }
}
img.LoadFromPixels(640, 480, camData); //Load pixels into SFML Image object for display

Unfortunately, this doesn't quite work. Something in that loop is wrong, as the resulting image when I load and display camData is scrambled.

As far as I can discern, either my math in the loop is wrong so the pixels are being assigned wrong, or the Mat data is in some format other than BGR.

Any ideas?

What exactly do you mean by scrambled? Could you maybe post an example of the resulting image? — sietschie, Apr 22 '12 at 06:32
[Example](http://i.imgur.com/dD2fk.png). The pixel data from the source image isn't ending up in the correct locations in the destination array, so it makes weird interleaving patterns. — Thew, Apr 22 '12 at 07:26
just had the same pattern after pulling data back from the gpu/opencl and dumping it to file :D — Mark Essel, Oct 24 '13 at 15:17

score 12 · Accepted Answer · answered Apr 22 '12 at 09:38

12

OpenCV can do all job for you:

VideoCapture cap(0);
Mat frame;
cap >> frame;

uchar* camData = new uchar[frame.total()*4];
Mat continuousRGBA(frame.size(), CV_8UC4, camData);
cv::cvtColor(frame, continuousRGBA, CV_BGR2RGBA, 4);
img.LoadFromPixels(frame.cols, frame.rows, camData);

answered Apr 22 '12 at 09:38

Andrey Kamaev

29,582
6
94
88

This LoadFromPixels method does not seem to be here anymore in OpenCV 4, is there a replacement ? – Jean-Michaël Celerier May 13 '22 at 12:29

score 3 · Answer 2 · answered Jan 20 '17 at 07:23

I like the accepted answer better but this snippet helps you understand what's going on.

 for (int i=0; i<srcMat.rows; i++) {
            for (int j=0; j<srcMat.cols; j++) {
                int index = (i*srcMat.cols+j)*4;
                // copy while converting to RGBA order
                dstRBBA[index + 0] = srcMat[index + 2 ];
                dstRBBA[index + 1] = srcMat[index + 1 ];
                dstRBBA[index + 2] = srcMat[index + 0 ];
                dstRBBA[index + 3] = srcMat[index + 3 ];
            }
        }

Go_Da · Answer 3 · 2015-02-11T12:17:49.470

For me worked following code:

VideoCapture capture(0);
Mat mat_frame;
capture >> mat_frame; // get a new frame from camera            

// Be sure that we are dealing with RGB colorspace...
Mat rgbFrame(width, height, CV_8UC3);
cvtColor(mat_frame, rgbFrame, CV_BGR2RGB);

// ...now let it convert it to RGBA
Mat newSrc = Mat(rgbFrame.rows, rgbFrame.cols, CV_8UC4);
int from_to[] = { 0,0, 1,1, 2,2, 3,3 };
mixChannels(&rgbFrame, 2, &newSrc, 1, from_to, 4);

The result (newSrc) is a premultiply image!

OpenCV 2.3: Convert Mat to RGBA pixel array

3 Answers3

Linked