Inconsistency rounding real numbers in C #

Question

I have this test code:

class Test
{
    static void Main()
    {
        decimal m = 1M / 6M;
        double d = 1.0 / 6.0;

        decimal notQuiteWholeM = m + m + m + m + m + m; // 1.0000000000000000000000000002M
        double notQuiteWholeD = d + d + d + d + d + d;   // 0.99999999999999989

        Console.WriteLine(notQuiteWholeM); // Prints: 1.0000000000000000000000000002
        Console.WriteLine(notQuiteWholeD); // Prints: 1.

        Console.WriteLine(notQuiteWholeM == 1M);  // False
        Console.WriteLine(notQuiteWholeD < 1.0);   // Prints: True. Why?

        Console.ReadKey();
    }
}

Why this line prints 1?

Console.WriteLine(notQuiteWholeD); // Prints: 1

an this one, why prints True?

Test class output

Is there an automatically rounding process? What can I do to print the correct/calculated value?

[Note: I found this example code in C# 5.0 in a Nutsheel page 30: Real Number Rounding Error].

Thanks in advance.

[What Every Computer Scientist Should Know About Floating-Point Arithmetic](http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) — NullUserException, Aug 06 '13 at 23:13
Remember that even "big decimals" are **arbitrary** precision, not **infinite** precision. In some cases, working with any finite representation will cause rounding, at best you can control when this occurs, as opposed to floats/doubles - or .NET decimals. — millimoose, Aug 06 '13 at 23:23

JimmiTh · Answer 1 · 2013-08-19T08:47:01.320

Not quite reading your question in the same way as the other two answers. The gist of it: Does the formatted string representation of a double "round" in C#?

Yes.

Internally double is represented with full IEEE-754 decimal digit precision (15-17 digits), which is why:

notQuiteWholeD < 1.0 == true    // because notQuiteWholeD = 0.99999999999999989

However, when formatting it as a string, by default it will use 15 digit precision - equivalent to:

String.Format("{0:G15}", notQuiteWholeD)   // outputs "1"

To get all the digits of the full internal representation, you can use:

Console.WriteLine("{0:G17}", notQuiteWholeD);

Or:

Console.WriteLine("{0:R}", notQuiteWholeD);

Both, in this case, will output "0,99999999999999989".

The former will always use 17 digit precision. The latter ("roundtrip precision") will use 15 digits if that's enough precision for the following to be true, otherwise it will use 17:

Double.Parse(String.Format("{0:G15}", notQuiteWholeD)) == notQuiteWholeD

Bonus Example: ... of when G17 and R differ:

Console.WriteLine("{0:G17}", 1.0000000000000699); // outputs "1.0000000000000699"
Console.WriteLine("{0:R}",   1.0000000000000699); // outputs "1.00000000000007"

1.0000000000000699 (17 significant digits) can be represented accurately enough for a roundtrip using only 15 significant digits. In other words, the double representation of 1.00...07 is the same as for 1.00...0699.

So 1.00...07 (15 digits) is a shorter input to get the exact same internal (17 digit) representation. That means R will round it to 15 digits, while G17 will keep all the digits of the internal representation.

Maybe it's clearer when realizing that this:

Console.WriteLine("{0:G17}", 1.00000000000007); // outputs "1.0000000000000699"
Console.WriteLine("{0:R}",   1.00000000000007); // outputs "1.00000000000007"

... gives the exact same results.

score 5 · Answer 2 · answered Aug 06 '13 at 23:20

Decimal is stored in terms of base 10. Double is stored in terms of base 2. Neither of those bases can exactly represent 1 / 6 with a finite representation.

That explains all the output except Console.WriteLine(notQuiteWholeD). You get "1" for output, even though the actual value stored is less than 1. Since the output is in base 10, it has to convert from base 2. Part of the conversion includes rounding.

score 3 · Answer 3 · edited May 23 '17 at 11:58

As we know, 1/6 = 0.1666 (repeating), decimal and double can not represent repeating numbers, they are calculated when assigned to. Since they are built from different backing data structures they represent a different set of possible numbers and round differently in some cases.

For this code:

Console.WriteLine(notQuiteWholeD < 1.0);   // Prints: True. Why?

Since notQuiteWholeD is 0.99999999999999989 it prints true.

I'm not going to cover how the double and decimal work behind the scenes but here is some reading material if you're interested.

Inconsistency rounding real numbers in C #

3 Answers3

Linked

Related