Correct YUV422 to RGB conversion

Question

I've been trying to tackle a YUV422 into a RGB conversion problem for about a week. I've visited many different websites and have gotten different formulas from each one. If anyone else has any suggestions I would be glad to hear about them. The formulas below give me an image with either and overall purple or a green hue in them. As of this moment I haven't been able to find a formula that allows me to get back a proper RGB image. I have include all my various chunks of code below.

    //for(int i = 0; i < 1280 * 720 * 3; i=i+3)
    //{
    //  /*m_RGB->imageData[i] = pData[i] + pData[i+2]*((1 - 0.299)/0.615);
    //  m_RGB->imageData[i+1] = pData[i] - pData[i+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[i+2]*((0.299*(1 - 0.299))/(0.615*0.587));
    //  m_RGB->imageData[i+2] = pData[i] + pData[i+1]*((1 - 0.114)/0.436);*/

    //  m_RGB->imageData[i] = pData[i] + 1.403 * (pData[i+1] - 128);
    //  m_RGB->imageData[i+1] = pData[i] + 0.344 * (pData[i+1] - 128) - 0.714 * (pData[i+2] - 128);
    //  m_RGB->imageData[i+2] = pData[i] + 1.773 * (pData[i+2] - 128);
    //}

    for(int i = 0, j=0; i < 1280 * 720 * 3; i+=6, j+=4)
    {
        /*m_RGB->imageData[i] = pData[j] + pData[j+3]*((1 - 0.299)/0.615);
        m_RGB->imageData[i+1] = pData[j] - pData[j+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[j+3]*((0.299*(1 - 0.299))/(0.615*0.587));
        m_RGB->imageData[i+2] = pData[j] + pData[j+1]*((1 - 0.114)/0.436);
        m_RGB->imageData[i+3] = pData[j+2] + pData[j+3]*((1 - 0.299)/0.615);
        m_RGB->imageData[i+4] = pData[j+2] - pData[j+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[j+3]*((0.299*(1 - 0.299))/(0.615*0.587));
        m_RGB->imageData[i+5] = pData[j+2] + pData[j+1]*((1 - 0.114)/0.436);*/

        /*m_RGB->imageData[i] = pData[j] + 1.403 * (pData[j+3] - 128);
        m_RGB->imageData[i+1] = pData[j] + 0.344 * (pData[j+1] - 128) - 0.714 * (pData[j+3] - 128);
        m_RGB->imageData[i+2] = pData[j] + 1.773 * (pData[j+1] - 128);
        m_RGB->imageData[i+3] = pData[j+2] + 1.403 * (pData[j+3] - 128);
        m_RGB->imageData[i+4] = pData[j+2] + 0.344 * (pData[j+1] - 128) - 0.714 * (pData[j+3] - 128);
        m_RGB->imageData[i+5] = pData[j+2] + 1.773 * (pData[j+1] - 128);*/

        BYTE Cr = pData[j+3] - 128;
        BYTE Cb = pData[j+1] - 128;
        /*m_RGB->imageData[i] = pData[j] + Cr + (Cr >> 2) + (Cr >> 3) + (Cr >> 5);
        m_RGB->imageData[i+1] = pData[j] - ((Cb >> 2) + (Cb >> 4) + (Cb >> 5)) - ((Cr >> 1) + (Cr >> 3) + (Cr >> 4) + (Cr >> 5));
        m_RGB->imageData[i+2] = pData[j] + Cb + (Cb >> 1) + (Cb >> 2) + (Cb >> 6);
        m_RGB->imageData[i+3] = pData[j+2] + Cr + (Cr >> 2) + (Cr >> 3) + (Cr >> 5);
        m_RGB->imageData[i+4] = pData[j+2] - ((Cb >> 2) + (Cb >> 4) + (Cb >> 5)) - ((Cr >> 1) + (Cr >> 3) + (Cr >> 4) + (Cr >> 5));
        m_RGB->imageData[i+5] = pData[j+2] + Cb + (Cb >> 1) + (Cb >> 2) + (Cb >> 6);*/

        /*int R1 = clamp(1 * pData[j] + 0 * Cb + 1.4 * Cr, 0, 255), R2 = clamp(1 * pData[j+2] + 0 * Cb + 1.4 * Cr, 0, 255);
        int G1 = clamp(1 * pData[j] - 0.343 * Cb - 0.711 * Cr, 0, 255), G2 = clamp(1 * pData[j+2] - 0.343 * Cb - 0.711 * Cr, 0, 255);
        int B1 = clamp(1 * pData[j] + 1.765 * Cb + 0 * Cr, 0, 255), B2 = clamp(1 * pData[j+2] + 1.765 * Cb + 0 * Cr, 0, 255);*/

        /*int R1 = clamp(pData[j] + 1.403 * (pData[j+3] - 128), 0, 255), R2 = clamp(pData[j+2] + 1.403 * (pData[j+3] - 128), 0, 255);
        int G1 = clamp(pData[j] + 0.344 * (pData[j+1] - 128) - 0.714 * (pData[j+3] - 128), 0, 255), G2 = clamp(pData[j+2] + 0.344 * (pData[j+1] - 128) - 0.714 * (pData[j+3] - 128), 0, 255);
        int B1 = clamp(pData[j] + 1.773 * (pData[j+1] - 128), 0, 255), B2 = clamp(pData[j+2] + 1.773 * (pData[j+1] - 128), 0, 255);*/

        int R1 = clamp((298 * (pData[j] - 16) + 409 * (pData[j+3] - 128) + 128) >> 8, 0, 255), R2 = clamp((298 * (pData[j+2] - 16) + 409 * (pData[j+3] - 128) + 128) >> 8, 0, 255);
        int G1 = clamp((298 * (pData[j] - 16) - 100 * (pData[j+1] - 128) - 208 * (pData[j+3] - 128) + 128) >> 8, 0, 255), G2 = clamp((298 * (pData[j+2] - 16) - 100 * (pData[j+1] - 128) - 208 * (pData[j+3] - 128) + 128) >> 8, 0, 255);
        int B1 = clamp((298 * (pData[j] - 16) + 516 * (pData[j+1] - 128) + 128) >> 8, 0, 255), B2 = clamp((298 * (pData[j+2] - 16) + 516 * (pData[j+1] - 128) + 128) >> 8, 0, 255);

        //printf("R: %d, G: %d, B: %d, R': %d, G': %d, B': %d \n", R1, G1, B1, R2, G2, B2);

        m_RGB->imageData[i] = (char)R1;
        m_RGB->imageData[i+1] = (char)G1;
        m_RGB->imageData[i+2] = (char)B1;
        m_RGB->imageData[i+3] = (char)R2;
        m_RGB->imageData[i+4] = (char)G2;
        m_RGB->imageData[i+5] = (char)B2;

        /*m_RGB->imageData[i] = (char)(clamp(1.164 * (pData[j] - 16) + 1.793 * (Cr), 0, 255));
        m_RGB->imageData[i+1] = (char)(clamp(1.164 * (pData[j] - 16) - 0.534 * (Cr) - 0.213 * (Cb), 0, 255));
        m_RGB->imageData[i+2] = (char)(clamp(1.164 * (pData[j] - 16) + 2.115 * (Cb), 0, 255));
        m_RGB->imageData[i+3] = (char)(clamp(1.164 * (pData[j+2] - 16) + 1.793 * (Cr), 0, 255));
        m_RGB->imageData[i+4] = (char)(clamp(1.164 * (pData[j+2] - 16) - 0.534 * (Cr) - 0.213 * (Cb), 0, 255));
        m_RGB->imageData[i+5] = (char)(clamp(1.164 * (pData[j+2] - 16) + 2.115 * (Cb), 0, 255));*/
    }

Any help is greatly appreciated.

What is the source of the YUV data, and what is the destination? For example if the destination is Windows you need to use BGR order rather than RGB. — Mark Ransom, Nov 07 '11 at 21:07
the YUV is coming from a Decklink Intensity Pro capture card. I also tried to flip the BGR/RGB values and it didn't help. This is being done on a Windows box — Seb, Nov 07 '11 at 21:15
If you are using the decklink SDK, why dont you simply use the ConvertFrame method which is a part of the API? — ronag, Nov 07 '11 at 21:24
It looks to me that you are confusing Cr with Cb, you should switch places. — ronag, Nov 07 '11 at 21:27
I tried to use ConvertFrame, but it throws an error when I use it. i contacted DeckLink and they said that YUV to RGB conversions are currently not supported — Seb, Nov 07 '11 at 21:34
cvCvtColor converts from YUV format and not YCrCb. At least that was the answer I've found when I looked around and even posted a question regarding it. — Seb, Nov 08 '11 at 13:20
At the time of writing, colour conversion in OpenCV does not support a colourspace newer than SD (aka Rec 601) !! So you can't get the exact colours when converting images using the colourspaces of the HD or UHD specs (Rec. 709 or Rec. 2020). — G Huxley, Dec 06 '16 at 21:57

ronag · Answer 1 · 2011-11-07T21:28:47.797

5

Some clues to help you along:

You are confusing Cr with Cb.

Assuming UYVY/422

Y1 = data[j+0];
Cr = data[j+1];
Y2 = data[j+2];
Cb = data[j+3];

Your conversion calculation are wierd, and incorrect for HD.

For SD

R = max(0, min(255, 1.164(Y - 16) + 1.596(Cr - 128)));
G = max(0, min(255, 1.164(Y - 16) - 0.813(Cr - 128) - 0.391(Cb - 128)));
B = max(0, min(255, 1.164(Y - 16) + 2.018(Cr - 128)));

For HD

R = max(0, min(255, 1.164(Y - 16) + 1.793(Cr - 128)));
G = max(0, min(255, 1.164(Y - 16) - 0.534(Cr - 128) - 0.213(Cb - 128)));
B = max(0, min(255, 1.164(Y - 16) + 2.115(Cr - 128)));

You could simply use ConvertFrame which is a part of the Decklink SDK.

edited Nov 07 '11 at 21:28

answered Nov 07 '11 at 21:12

ronag

49,529
25
126
221

thanks as your answer helped me. Questions: HD means 1280x720 or 1920x1080? what about 4k? btw the order Y1CrY2Cb worked for me with yuy2! although based on msdn Cb is 'u' and it was supposed to be Y1CbY2Cr but doing that shows blue in place of red! can you explain why the order twisted? or does msdn Cb='u' not valid?1 – Jawad Al Shaikh Jul 30 '15 at 07:06
1

Err... shouldn't the Cr for B be Cb? – Hugo Maxwell Sep 12 '19 at 17:29
Echoing Hugo's comment, Equation for B needs to be corrected in both SD and HD, should be Cb not Cr – Christian Davis Apr 19 '20 at 17:39

score 3 · Answer 2 · answered Dec 23 '11 at 12:42

Your problem is that there are lots of YUV422 formats out there. You must find the exact one (the FOURCC index for the specific video you're using), and then figure out the correct way to decode it.

What you can do is to save some video from your board, open it in VLC, and look at the Codec details to find the exact FOURCC used.

http://www.fourcc.org/yuv.php

score 1 · Answer 3 · answered Nov 07 '11 at 21:16

Assuming packed 422 I don't see any of your blocks sampling the input data correctly. In packed 422 the input data will go Y1U1Y2V1 Y3U2Y4V2 where the overall image is a Y (luma) image at full resolution and one each of U and V each at half horizontal resolution.

Here's where I would start: Unpack alternating values of the input and extract a grayscale image:

for (uint i = 0, j = 0; i < 1280 * 720 * 3; i += 3, j += 2) {
    m_RGB->imageData[i] = pData[j];
    m_RGB->imageData[i+1] = pData[j];
    m_RGB->imageData[i+2] = pData[j];
}

Once you have that tuned to produce a grayscale image then introduce U and V by looking at pData[j+1] and pData[j+3] (or, on even pixels, pData[j-1] and pData[j+1]). Simplifying that is why some algorithms do two YUV pixels at a time.

When that works consider extracting the U and V images and properly resampling them to full resolution to produce a 444 image. Simply duplicating U and V for adjacent pixels is like upscaling by duplicating pixels.

(Note that other arrangements like 420 have even more complicated co-siting)

This got me a grayscale image, but everything looks as if a Gaussian blur was applied to it. — Seb, Nov 07 '11 at 21:41

Jo VdB · Answer 4 · 2011-12-23T12:35:56.870

0

I also struggled with the conversion

// Get the bytes
var u = bytes[0]; 
var y1 = bytes[1];
var v = bytes[2];
var y2 = bytes[3];

// Convert, cast to signed byte is important!
var r = y + (1.403 * (sbyte)v);
var g = y - (0.344 * (sbyte)u) - (0.714 * (sbyte)v);
var b = y + (1.770 * (sbyte)u);

if (r < 0)
    r = 0;
else if (r > 255)
    r = 255;

if (g < 0)
    g = 0;
else if (g > 255)
    g = 255;

if (b < 0)
    b = 0;
else if (b > 255)
    b = 255;

return Color.FromArgb((byte)r, (byte)g, (byte)b);

u and v are sbyte, and y is just a byte.

edited Dec 23 '11 at 12:35

answered Dec 13 '11 at 22:54

Jo VdB

2,016
18
16

Are you saying this is the correct conversion or that you struggled trying to use this conversion? – Olli Niemitalo May 27 '16 at 06:31

Correct YUV422 to RGB conversion

4 Answers4

Linked