Comparison is always false due to limited range of data type trying to saturate colors

Question

I'm trying to saturate my colors and make sure they do not overflow so that they will draw a pretty Mandlebrot set without being pixelated. I'm using an Altera DE2 board to try and print this Mandlebrot set via VGA connection to a computer screen and the colors are slightly off (pixely).

How exactly do I correct the if statements in the code below to not always be false?

if (iteration == 500)
{
    alt_up_pixel_buffer_dma_draw(my_pixel_buffer,0,0,0);
}
else
{
    //double z = sqrt(xtemp * xtemp + y * y);
    //int brightness = 256. * log2(1.75 + i - log2(log2(z))) / log2(500);
    //color(brightness, brightness, 255);
    //color is some function of iteration
    alt_u8 Red = (iteration*8);///zoom);
    if(Red > 255) // this if statement is always false
        Red = 255;

    alt_u8 Green = (iteration*4);///zoom);
    if(Green > 255) // this if statement is always false
        Green = 255;

    alt_u8 Blue = (iteration*2);///zoom);
    if(Blue > 255) // this if statement is always false
        Blue = 255;

    //draw the pixels
    alt_up_pixel_buffer_dma_draw(my_pixel_buffer, (Blue) + (Green<<8) + (Red<<16),j,i);
}

sorry, thank you. I'm looking to possibly understand why or how to fix the if statements in the code to not always yield a false — Coltstick, Apr 15 '16 at 23:44

score 1 · Accepted Answer · answered Apr 15 '16 at 23:46

1

You need to use a larger integer to get the result of the multiplication, so you can test if it exceeds the limit.

alt_u8 Red;
uint16_t tempRed = iteration * 8;
if (tempRed > 255) {
    Red = 255;
} else {
    Red = tempRed;
}

answered Apr 15 '16 at 23:46

Barmar

741,623
53
500
612

Jonathan Leffler · Answer 2 · 2016-04-16T00:09:43.787

Since the maximum value that can be stored in an alt_u8 is 255, there's no point checking whether it holds a larger value. (You would not be getting the warning if that were not the case, even though the type of alt_u8 is not visible.)

However, it also means that your assignments are dangerous because the assigned value is stored modulo 256. You probably need to use:

alt_u32 new_Red = iteration * 8;  // Guessed type; uint32_t or uint16_t would do
alt_u8 Red = new_Red;
if (new_Red > 255)
    Red = 255;

Now the value in new_Red can exceed 255, but the assignments ensure the saturated value is stored, rather than new_Red % 256 which would be stored otherwise.

Comparison is always false due to limited range of data type trying to saturate colors

2 Answers2