3

I read about a rounding technique at this question. Converting color value from float 0..1 to byte 0..255

You have a float on the range from 0 to 1 and you want to map it to the byte integer range of 0-255. So you multiply the 0-1 number by 255f, and then convert that number to a byte. Because all conversions just floor the decimal part of the float you need a way to make it so when it is floored it will achieve the desired rounded result.

My idea is to just add 0.5f before it is converted into a byte.

However, I noticed something new that I don't understand on that page I linked. He maps 0-1 to 256 instead, and this somehow will achieve a rounded result.

I tested values to see if it works and it appears it does. I just don't understand why. Can someone show me a proof?

Community
  • 1
  • 1
Dan Webster
  • 1,185
  • 1
  • 12
  • 27

2 Answers2

6

It is a special case of the Fencepost error. The easiest explanation can you see in the case that you have only two colors, black and white. It is therefore easy to see that you must divide the interval 0-1 by two.

This can be easily achieved by multiplying the interval with a slightly less number for division and truncating the result. This is also the reason that random number generators return a value >= 0 and < 1.0 to get easily values with equal probability.

So in this example you get the correct result with value = floorf(1.9999*x).

The problem is that while the color index ends with the number 1, there are actually two colors (0 and 1) and therefore the interval must be divided by two. With 256 colors (0-255) you need therefore divide by the number 256. People are likely to oversee that 0 is also a valid value and must be mapped correctly.

Thorsten S.
  • 4,144
  • 27
  • 41
3

Simply put, if your were representing colors with the integers "0, 1, 2, 3", you'd need to pick something to be your 0.5 case. If you pick the 1, you'd get 0.0, 0.5, 0.75, 1.0. If you pick the 2 you'd get 0.0, 0,25, 0.5, 1.0. There's no way to make those intervals equal, but integer colors can't have a decimal. The only solution would be to have eliminated the 0.5 case entirely and used even intervals, 0.333 0.666, but they chose not to do that. So color 127/255 currently evaluates to 0.5, instead of 0.498, because it's taken as 128/256. Unfortunately, adding 1 to the numerator and denominator would cause color 0 to be interpreted as 1/256 instead of 0/255

redux
  • 1,157
  • 10
  • 21
  • 1
    My statement was factual and on topic. If someone has a problem with the answer, try commenting rather than down voting. – redux Sep 11 '14 at 16:58