Commutative property of addition with double precision numbers

Question

Consider the following Unit Test:

// Works (sum 0.1 to 0.4)
float f1 = 0.1F + 0.2F + 0.3F + 0.4F;
Assert.AreEqual(1F, f1);

// Works too (sum 0.4 to 0.1)
float f2 = 0.4F + 0.3F + 0.2F + 0.1F;
Assert.AreEqual(1F, f2);

// Works (sum 0.1 to 0.4)
double d1 = 0.1D + 0.2D + 0.3D + 0.4D;
Assert.AreEqual(1D, d1);

// Fails! (sum 0.4 to 0.1)
double d2 = 0.4D + 0.3D + 0.2D + 0.1D;
Assert.AreEqual(1D, d2);

Everything works as expected for the float type (the sum is 1 in both cases), but when using double, the commutativity of the addition is not honored. Indeed, the sum of the first one is 1, but for the second I get 0.99999999.....

I understand why the result is once 1 and once not (because some numbers cannot be represented without loss of precision in base 2) but this does not explain why it works for float and not for double...

Could someone explain this?

In general, one shouldn't expect *associativity* with floating point. However, as to why your test behaves the way it does, I'm not sure. — Oliver Charlesworth, May 28 '13 at 08:40
It's well known that you get different results depending on the order in which you add floating point numbers. In particular, for maximum accuracy you should sort and add them from smallest to largest to minimise rounding errors. Also see [this interesting thing](http://en.wikipedia.org/wiki/Kahan_summation_algorithm) and [this SO question](http://stackoverflow.com/questions/6699066/in-which-order-should-floats-be-added-to-get-the-most-precise-result) — Matthew Watson, May 28 '13 at 08:53

score 3 · Answer 1 · answered May 28 '13 at 08:44

3

Have a look at the below

        // This works (sum 0.1 to 0.4)
        double d1 = 0.1D + 0.2D + 0.3D + 0.4D;
        double d11 = 0;
        d11 += 0.1D;//0.1
        d11 += 0.2D;//0.30000000000000004
        d11 += 0.3D;//0.60000000000000009
        d11 += 0.4D;//1.0

        // This does NOT work! (sum 0.4 to 0.1)
        double d2 = 0.4D + 0.3D + 0.2D + 0.1D;
        double d22 = 0;
        d22 += 0.4D;//0.4
        d22 += 0.3D;//0.7
        d22 += 0.2D;//0.89999999999999991
        d22 += 0.1D;//0.99999999999999989

And debug, look at the individual steps.

What you need to remember is that

        double d2 = 0.4D + 0.3D + 0.2D + 0.1D;

can also be seen as

        double d2 = (((0.4D + 0.3D) + 0.2D) + 0.1D);

The problem seems to not be 2 representations of the number 1, but more 2 paths to how it got there.

answered May 28 '13 at 08:44

Adriaan Stander

162,879
31
289
284

Buy the question is why does this differ between single and double precision? – Oliver Charlesworth May 28 '13 at 08:45
2

Well, float and double have different rounding errors so they can give different results. It really is pretty much as simple as that. – Matthew Watson May 28 '13 at 08:55
Another way to look at it is that adding two numbers will generally yield a rounding error proportional to the larger number; if one adds 0.1 to 0.2 to 0.3 to 0.4, only one addition will involve a number larger than 0.5. If one adds 0.4 to 0.3, and then adds that result to 0.2, and adds that result to 0.1, two of the additions will involve a number larger than 0.5. Also, the commuted form of 0.1+0.2+0.3+0.4 would be 0.4+(0.3+(0.2+0.1)), which would likely equal 0.1+0.2+0.3+0.4. – supercat Jul 05 '14 at 20:16

score 2 · Accepted Answer · answered May 28 '13 at 09:01

float f11 = 0;
f11 += 0.1F;//0.1
f11 += 0.2F;//0.3
f11 += 0.3F;//0.6
f11 += 0.4F;//1.0

float f2 = 0.4F + 0.3F + 0.2F + 0.1F;
float f22 = 0;
f22 += 0.4F;//0.4
f22 += 0.3F;//0.700000048
f22 += 0.2F;//0.900000036
f22 += 0.1F;//1.0

To add to astander's answer - this is how values looks for floats. Due to lower precision (7 digits for floats, 14-15 for doubles) values ends up being displayed differently and accidentally equal to what you expected.

But that's it - it's just coincidence! Never depend on it. Floating point operations are associative, nor precise. Never compare floats or doubled using ==, always consider using some margin value. This sample works for 1, but for other value it will surly fail.

score 1 · Answer 3 · answered May 28 '13 at 09:07

In the following:

float f = 0.3F + 0.3F + 0.2F + 0.1F;
double d = 0.3D + 0.3D + 0.2D + 0.1D;

The result will be:

float f = 0.900000036f;
double d = 0.9;

So for different numbers rounding errors might occur in floats where there are none in doubles and vice versa - that's because they have different number of digits thus the place where the rounding errors might occur differs.

score 0 · Answer 4 · answered May 28 '13 at 10:35

This is a known problem in comparing floating point numbers because according to C# specification they are implemented based on a nasty IEEE standard that causes this behavior.

So you never should compare 2 float or double in C#. Instead you should see if their difference is less than a specific delta value.

Commutative property of addition with double precision numbers

4 Answers4