Correct RGB values for AVFrame

Question

I have to fill the ffmpeg AVFrame->data from a cairo surface pixel data. I have this code:

/* Image info and pixel data */
width  = cairo_image_surface_get_width( surface );
height = cairo_image_surface_get_height( surface );
stride = cairo_image_surface_get_stride( surface );
pix    = cairo_image_surface_get_data( surface );

for( row = 0; row < height; row++ )
{
    data = pix + row * stride;
    for( col = 0; col < width; col++ )
    {
        img->video_frame->data[0][row * img->video_frame->linesize[0] + col] = data[0];
        img->video_frame->data[1][row * img->video_frame->linesize[1] + col] = data[1];
        //img->video_frame->data[2][row * img->video_frame->linesize[2] + col] = data[2];
        data += 4;
    }
    img->video_frame->pts++;
}

But the colors in the exported video are wrong. The original heart is red. Can someone point me in the right direction? The encode.c example is useless sadly and on the Internet there is a lot of confusion about Y, Cr and Cb which I really don't understand. Please feel free to ask for more details. Many thanks.

Can you please post a complete code sample - something we can compile and execute? You may have to be creative - consider posting a 8x8 input image as a C array (initialized by code). Can you describe the data arrangement of `pix` (is it row major RGBA with `width`, `height` and `stride`)? In which part YCbCr pixel format is used? — Rotem, Dec 06 '21 at 17:21
Thank you so much for replying. I do appreciate. The code is a part of a GTK3 GUI which turns a series of pictures into a movie. It's a project I have been developing since 2009. I can't provide a code snippet sadly but the software is opensource. The cairo surface is RGB24. The AV context is created as AV_PIX_FMT_YUV420P because if I use AV_PIX_FMT_RGB24 I get the error message during the enconding: "Input width is greater than stride". Hope this helps, please feel free to ask for more info. — GiuTor, Dec 06 '21 at 17:58
My current understanding of YCbCr is that it is made of Luminance/Red-diff/Blue-diff where Luminance is brightness, and Red-diff and Blue-diff are chroma values with information about Red/green and Blue/green respectively. It works by combining a black and white image with the two chroma channels — Base64__, Dec 15 '21 at 02:49

score 1 · Answer 1 · answered Dec 10 '21 at 08:58

1

You need to use libswscale to convert the source image data from RGB24 to YUV420P.

Something like:

int width  = cairo_image_surface_get_width( surface );
int height = cairo_image_surface_get_height( surface );
int stride = cairo_image_surface_get_stride( surface );
uint8_t *pix = cairo_image_surface_get_data( surface );

uint8_t *data[1] = { pix };
int linesize[1]  = { stride };

struct SwsContext *sws_ctx = sws_getContext(width, height, AV_PIX_FMT_RGB24 ,
                                            width, height, AV_PIX_FMT_YUV420P,
                                            SWS_BILINEAR, NULL, NULL, NULL);

sws_scale(sws_ctx, data, linesize, 0, height, 
          img->video_frame->data, img->video_frame->linesize);

sws_freeContext(sws_ctx);

See the example here: scaling_video

answered Dec 10 '21 at 08:58

aergistal

29,947
5
70
92

Thanks for your reply. I also used sws_scale, no errors during the encoding but the video doesn't even play and VLC shows a bunch of these useless messages in the terminal window: [h264 @ 0x7f1508c82700] co located POCs unavailable [h264 @ 0x7f1508c9f400] mmco: unref short failure [h264 @ 0x7f1508cd8f80] co located POCs unavailable The example wasn't helpful as it convert FROM YUV420P but thanks I appreciated it. If you would like to know more I can commit the changes to the SVN server and you can give a look, it's a nice GUI and a nice software, just underrated sadly. – GiuTor Dec 11 '21 at 09:53
@GiuTor Could you dump a sample cairo image to a file somewhere and post the format, width & height? – aergistal Dec 11 '21 at 18:29
is this code correct for dumping the image: /* Image info and pixel data */ width = cairo_image_surface_get_width( surface ); height = cairo_image_surface_get_height( surface ); stride = cairo_image_surface_get_stride( surface ); pix = cairo_image_surface_get_data( surface ); FILE *fp = fopen("cairo_image.raw","w"); fwrite (pix, 4*width, sizeof(pix), fp); fclose(fp); – GiuTor Dec 12 '21 at 20:23

Correct RGB values for AVFrame

1 Answers1