C# float precision

Question

Can anyone please explain to me what's happening here:

using System;
using System.Text;

namespace ConsoleApplication1 {

    class Program {

        static void Main(string[] args) {

            object o = 1000000.123f;
            float f= Convert.ToSingle(o);
            double d = Convert.ToDouble(f);

            Console.WriteLine(f.ToString("r"));
            Console.WriteLine(d.ToString("r"));

            Console.ReadLine();

        }
    }
}

Which outputs:

1000000.13

1000000.125

I expected:

The object o to have an underlying float type (seems to happen [from observing the watch window where it is typed as object {float})

That 1000000.123f would get stored in f as 1000000.125 (The IEEE754 approximation in 32 bits?)

That the double would store 1000000.125 as well (seems to happen even though f doesn't seem to contain what I expected)

That asking for a round trip format on the ToString would give me back 1000000.125 in both cases.

Can anyone tell me what I'm doing wrong to get 1000000.13 when stringing out f?

Marcelo Cantos · Accepted Answer · 2014-02-10T21:55:24.783

6

As you have already observed, the number 1000000.123 is stored as 1000000.125. This is rendered as-is by double.ToString(), but truncated by float.ToString() because showing too many digits is misleading.

Incidentally, there is no Convert.ToSingle(float) because it would simply return exactly what you passed in. Your code is actually resolving to Convert.ToSingle(double). You are thus (implicitly) converting to double and then (explicitly) back to float, which is a no-op, essentially.

Caution: Don't trust JavaScript floating point calculators. Some of them assert that 1000000.123 is stored as 1000000.1 by single-precision floats, which I'm guessing is based on the assumption that, because IEEE floats have roughly 7.22 digits of precision, they can be accurately represented in 8 digits. This is incorrect.

edited Feb 10 '14 at 21:55

answered Feb 10 '14 at 21:33

Marcelo Cantos

181,030
38
327
365

thanks for the answer (and the heads up re: my conversion nop). Has the float.ToString() not rounded the number though? Wouldn't a truncation result in 1000000.12? – Stephen Emm Feb 10 '14 at 22:11
@StephenEmm: When it's halfway between two representations, it rounds up (or away from zero — not sure which — but it makes no difference for positive numbers). – Marcelo Cantos Feb 11 '14 at 09:06

C# float precision

1 Answers1

Linked