0

Let me preface the question by saying that I understand why values such as 0.1, 3.14, 0.2, and other values not composable of combinations of powers of two are ultimately unrepresentable by IEEE-754 formats, and that they may only be approximated as best as the precision allows.

What I am having trouble understanding is why attempting to represent the value 2-23 results in a slight margin of error.

2-23 is exactly equal to 1.1920928955078e-7, or 0.00000011920928955078. In single-precision IEEE-754, it can be constructed as follows:

  • The sign bit is 0
  • The biased exponent is 104 (or 0b01101000 in binary) to account for the 127-bias, leading to -23 being the final exponent value
  • The mantissa's bit field is entirely composed of 0s, its ultimate value being 1.0 when the implicit 1-bit is accounted for

However, storing this particular bit sequence in memory and printing it out in decimal notation, with 25 digits of precision past the decimal point results in the following:

0.0000001192092895507812500
                      ^
                      |
                      margin of error starts here

This value contains an error of exactly 1.25e-21. On this interactive website, this error value is referred to as an "Error due to conversion".

I am having trouble grasping this - because I understand, for example, why a value such as +3.14 cannot be exactly represented by a single-precision bitfield. No combination of negative powers of two in the mantissa scaled by the value in the exponent can exactly represent 3.14, so the next closest approximation is chosen. Thus an "error due to conversion" is expected. Contrary to that, the value 2-23 is able to be stored exactly in a single-precision bitfield, yet when converted back to a decimal notation, an error appears.

There's clearly some sort of misunderstanding on my part, but I can't figure out where exactly.

bool3max
  • 2,748
  • 5
  • 28
  • 57

1 Answers1

1

2-23 is in fact not equal to exactly 1.1920928955078e-7.

The software I was using to perform the calculation was truncating the result three digits short... The exact value of 2-23 is precisely 0.00000011920928955078125, or 1.1920928955078125e-7, which is the output I was observing before, making it correct, with no margin of error whatsoever.

It is also worth nothing that the website truncates the decimal representation of the mantissa a few digits early, making by-hand calculations (i.e. multiplying the mantissa with the sign bit and 2exponent) seem incorrect (or at least slightly off) when they are in fact not.

bool3max
  • 2,748
  • 5
  • 28
  • 57
  • 1
    The `decimal` package in Python can be nice for seeing decimal expansions of (double precision) floats. For example `decimal.Decimal(2**-23)` gives me the full expansion, or as a command line tool: `python -c "from decimal import Decimal as D; print(str(D(2**-23)))"` – Sam Mason Sep 27 '21 at 08:07