0

I've been writing a decimal to single-precision IEEE754 floating-point converter, and I found a few discrepancies with numbers like 0.1 and 0.2

Let's take 0.1, the first step would be converting it into a simple binary representation (multiplying the fractional part by 2 and taking the integral part) This gives me a well-known recurring binary pattern (001100110011...)

The final binary representation of 0.1 (up to 26-bits) is 0 . 00 0110 0110 0110 0110 0110 0110.

To fit it into a 32-bit floating-point number, the next step is to normalize it by shifting the decimal point 4 times to the right, removing the leading 1., and truncating it to 23 bits. This leaves me with 10011001100110011001100. Most programming languages give me 10011001100110011001101 (with the last bit 1 instead of 0). What am I doing wrong here?

Some programmer dude
  • 400,186
  • 35
  • 402
  • 621
Soumil
  • 101
  • 1
  • 1
    Carry? Rounding? What is the actual floating-point value of your `10011001100110011001100` versus the others `10011001100110011001101`, which is closer to the actual `0.1`? – Some programmer dude Apr 14 '22 at 06:10
  • @Someprogrammerdude the actual value of my number is 0.0999999940395355224609375, which is apparently further from the actual 0.1 than with the last bit flipped. I still have no idea _how_ it gets flipped though. 0011 is the infinitely recurring sequence here, so logically 00 should come after 11, not 01. – Soumil Apr 14 '22 at 09:39
  • Show the code you are using to do these calculations. Edit the question to provide a [mre]. – Eric Postpischil Apr 14 '22 at 11:34

1 Answers1

4

... removing the leading 1., and truncating it to 23 bits.

Wrong operation: round, not truncate. Wrong order of operations - round, then remove the leading 1.

... still have no idea how it gets flipped though

The infinite exact binary answer rounded to the closest float.

With common float32, the significand is 24-bit*1. Conversion of values with more than 24-bits is usually a round*2, not truncate. The remaining bits were 110_0110_...

        123 4567 8901 2345 6789 0123 4
0 . 00 0110 0110 0110 0110 0110 0110 0110 0110 ...
        ^----------------------------^
0 . 00 0110 0110 0110 0110 0110 0110 1 rounded
         10 0110 0110 0110 0110 0110 1 encoded 23-bits

The rounding happens before the leading bit is removed as part of the encoding as the most significant bit place may change.

Notes:

0.000110011001100110011001100           (binary) = 0.0999999940395355224609375000 (decimal)
0.0001100110011001100110011001100110... (binary) = 0.1                            (decimal)
0.000110011001100110011001101 (closer)  (binary) = 0.1000000014901161193847656250 (decimal)

*1 Most significant 1 bit implied, 23-bits explicitly coded.
*2 Round to nearest, ties to even.

chux - Reinstate Monica
  • 143,097
  • 13
  • 135
  • 256