2

From "Image Format", we see:

An 11-bit float has no sign-bit; it has 6 bits of mantissa and 5 bits of exponent.

From "Half-precision floating-point format", we can deduct that: The relative precision of 11-bit float is 2^(-6), which is 1/64

But for a 8-bit RGB image, the relative precision at 255, is (255-254)/255 = 1/255.

So does it say the precision of GL_R11F_G11F_B10F is not good for 8bit RGB images at a brightness range which is large (for example, at 255)?

user1914692
  • 3,033
  • 5
  • 36
  • 61
  • It's odd that you linked to the OpenGL wiki article on image formats, but didn't notice the article that [explains exactly how much precision](https://www.opengl.org/wiki/Small_Float_Formats#Numeric_limits_and_precision) you get from such floats. – Nicol Bolas Apr 29 '16 at 14:49
  • @Nicol Bolas, I think the relative precision matters. So that was the reason I didn't include the article you cited on "Decimal digits of precision" – user1914692 Apr 29 '16 at 15:30
  • But the "Decimal digits of precision" *is* the "relative precision". It is the number of digits of precision that you get from the mantissa, regardless of the exponent. – Nicol Bolas Apr 29 '16 at 15:32
  • @Nicol Bolas, Thanks for your discussion. We can use "Decimal digits of precision" to deduce the relative precision. But I think they are two different concepts. To hep me decide whether we can have precise 5 interval from 250-255, we need to use the concept of "relative precision". If we have 6 bits for Significand precision, one interval is 2^(-6). And if I have value 1, and the next value is 1 + 2^(-6). Is my understanding correct? Thanks. – user1914692 Apr 29 '16 at 15:43

1 Answers1

3

The reason for using a floating-point image is to allow you to express values outside the range [0, 1]. If that's what you need to do, then that's what you need to do, and normalized integer formats will not be helpful, regardless of their precision.

This particular floating point format compacts floating-point RGB data down as far as possible without using a shared exponent. It does lose precision relative to normalized 8-bit integers, having only about 2.1 digits of decimal precision (because of the way IEEE-754 floats work, you get the effect of having one more bit of mantissa than you actually have).

But as previously stated, if you need values greater than 1, normalized formats are not an option. This particular floating-point format is for cases where the loss of precision is tolerable. This would be for uses like framebuffer outputs in HDR cases. It's half the size of 16-bit float RGBA formats, so that can represent a non-trivial performance improvement on framebuffer bandwidth costs.

It's an optimization. You might use half-floats for development, but then you switch to this and see if you can tell the difference. If not, you switch to it permanently (possibly with a user-facing setting for higher image fidelity).

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
  • Since we care about relative precision, error/value, I think it doesn't matter whether we use 0-255 or 0-1 in GL_R11F_G11F_B10F format. I just want to seek confirmation, or rebuttal, that for pixel with intensity around 250-255 (in the 0-255 range) or 0.98-1 (in the 0-1) arrange, it cannot have precision in GL_R11F_G11F_B10F format as in 8bit integer expression. That means, in 8bit integer expression, we can have 250, 251, ... 255. But I think there are no 6 steps of value expression in GL_R11F_G11F_B10F format . Is my understanding correct? – user1914692 Apr 29 '16 at 15:34
  • I thought "*It does lose precision relative to normalized 8-bit integers*" was pretty clear. Furthermore... why do you care so much? As I pointed out, if you need floating-point values, then normalized integers aren't even on the table as a possibility. And if you can live within [0, 1], why not just use normalized integers? The main point of my answer is that you're asking the wrong question. You're starting from the wrong place. – Nicol Bolas Apr 29 '16 at 15:48
  • Thanks. I will check your answer. Actually I will do some mathematical operation on the pixel values. So it might be greater than 1 (which can be met by GL_R11F_G11F_B10F format), and also might need higher precision than 0-255 levels (this is the question I asked). Thanks again. – user1914692 Apr 29 '16 at 16:10
  • BTW, I think "relative error" is what really matters.See the section "Precision limitations on decimal values in [0, 1]" in https://en.wikipedia.org/wiki/Half-precision_floating-point_format For example, it states: Decimals between 2^(−11) and 2^(−10): fixed interval 2^(−21), this is what is concerned. – user1914692 Apr 29 '16 at 19:51
  • Did you notice the fact that `-21 = -10 - 11`? Where -10 is the largest exponent in that range, and 11 is the number of bits of precision in the significand? Every (non-denormalized) interval works with that exact math. Even [1, 2]: `-10 = 1 - 11`. In short, what you're talking about is *no different* from saying that you get 11 bits of precision in the significand. – Nicol Bolas Apr 29 '16 at 20:06
  • Honestly, I noticed that yesterday night :) The concept you used "decimal digits" might mislead to understand your post; so I didn't understand much and didn't accept it before. So the relative error is always the same, say for GL_R11F_G11F_B10F, it is always 1/2^6. But, the relative error for a equally spaced intensity values is not constant. – user1914692 Apr 29 '16 at 20:23
  • (to be continued), So for 5/255, jumping one level, to 6/255, the relative error is 1/5. For 240/255, jumping one level, to 241/255, is 1/240. But for GL_R11F_G11F_B10F, it always has teh same relative error, so this means, it expresses 5/255 more actually than 240/255. Or in another way, for the range 5/255-10/255, it has more intervals than the range 240/255 - 245/244. Sorry for my input. I didn't mean to disagree with you. I just want to express in a different way to make understanding more clear. – user1914692 Apr 29 '16 at 20:23