Can we assume any pair of following floating point arithmetic statements always produce identical result?

Question

Given following floating points variables with arbitrary value, in c/c++.

float a, b, c, d;

In following statements, can we assume any pair of them would always generate identical result?

float result_1 = a + b + c + d - c;
float result_2= a + b + c + (d - c);
float result_3 = a + b + d;

Also, is there any guarantee on following predicate:

a + b - b == a

Plus you can get different results depending on the optimization settings — Luis, Jul 24 '14 at 20:32
@TonyHopkinson You can go quite far nowadays assuming the IEEE 754 standard is implemented correctly for the basic operations. — Pascal Cuoq, Jul 24 '14 at 20:52
@PascalCuoq, yeah far enough to find out you should have used a fixed point type, again... — Tony Hopkinson, Jul 25 '14 at 20:03
@TonyHopkinson I am sure that all operations are exact in fixed-point, starting with multiplication. Oh wait, no, multiplication is not exact at all in fixed-point. It seems there is no silver bullet to the problem of representing an infinity of values in a finite number of bits. But at least one solution requires you to evaluate the optimal shift and width of all intermediate results at design-time, lest it be horribly wasteful! And that solution is… fixed-point. — Pascal Cuoq, Jul 25 '14 at 20:17

score 4 · Accepted Answer · answered Jul 24 '14 at 20:39

4

No, you can not assume this. I broke all three of your examples: (Live)

#include <iostream>

int main()
{
   double a = 1, b = 1e100, c= 1e100, d= 1, c2 = .1, d2 = -.1, b2 = 1;

   std::cout << ( a + b2 + c2 + d2 - c2 == a + b2 + c2 + (d2 - c2)) << "\n"
    << ( a + b2 + c + d - c == a + b2 + d) << "\n"
    << ( a == a + b -b);
}

Output:

0 0 0

== and != are always unsafe on floating point types because the have rounding errors.

answered Jul 24 '14 at 20:39

Baum mit Augen

49,044
25
144
182

2

“== and != are always unsafe on floating point types because the have rounding errors” Actually, `==` and `!=` are very useful floating-point operations, and two of the few operations that are exact. – Pascal Cuoq Jul 24 '14 at 20:45
3

@PascalCuoq _'and two of the few operations that are exact'_ Yeah, the only problem is: You rarely have _exact_ values to compare, besides comparing known constants (which renders at least useless). – πάντα ῥεῖ Jul 24 '14 at 20:49
@πάνταῥεῖ So we agree that +, -, 0.1 and 1e100 are “always unsafe on floating-point types” and that `==` and `!=` are fine, then. – Pascal Cuoq Jul 24 '14 at 20:51
1

@PascalCuoq I tend to use some comparison involving [`std::numeric_limits::epsilon`](http://en.cppreference.com/w/cpp/types/numeric_limits/epsilon), to catch up such problems. – πάντα ῥεῖ Jul 24 '14 at 20:56
@πάνταῥεῖ I use `==` all the time: http://blog.frama-c.com/index.php?post/2013/09/25/The-problem-with-differential-testing-is-that-at-least-one-of-the-compilers-must-get-it-right http://blog.frama-c.com/index.php?post/2013/05/01/A-conversionless-conversion-function2 – Pascal Cuoq Jul 24 '14 at 21:01
2

@PascalCuoq I think [I've seen this trick already today](http://stackoverflow.com/questions/24937085/reliably-using-doubles-as-stdmap-keys#comment38752077_24937085). Still find it questionable to be the right approach for the problem (by design). There's a generalization of Fowler's Money pattern I mentioned, that's called [Quantity](http://martinfowler.com/eaaDev/quantity.html). IMHO using such is much more expressive, than using obscure casting trickery with floating point values (and doubt it's less efficient, if done right). – πάντα ῥεῖ Jul 24 '14 at 21:19
2

@πάνταῥεῖ This question is not about monetary amounts, and I have no idea why you bring up this remark. – Pascal Cuoq Jul 24 '14 at 21:31
1

Why the downvote? I proved that OP's assumption does not hold, thus the question is correctly answered. Please explain. – Baum mit Augen Jul 24 '14 at 21:33
@PascalCuoq _'This question is not about monetary amounts'_ No it isn't. What I wanted to express was: _Either you don't care about small differences, then deal with floating point and epsilon, or even the smallest difference needs to be fixed up for your problem, then don't use floating point but fixed point values_. Hope this makes my point clearer for you. – πάντα ῥεῖ Jul 24 '14 at 21:40
1

@πάνταῥεῖ Again, who said anything about “fixing up” apart from you? You brought up the example from another question. I said that `==` and `!=` were useful operations that were included in the language for a reason, and I gave a couple of examples that have nothing to do with “fixing up” anything (you are welcome to try to solve the same problems with the superstitious approach to floating-point that you describe and with std::numeric_limits::epsilon) – Pascal Cuoq Jul 24 '14 at 21:50
2

@BaummitAugen You said that “== and != are always unsafe”, which is wrong. You are free to believe what you want, but transmitting the superstition is harmful in my opinion. I have already commented to this effect. – Pascal Cuoq Jul 24 '14 at 21:54
@PascalCuoq _'who said anything about “fixing up” apart from you?'_ Yeah, maybe I'm sometimes thinking too far on such questions. But I always ask myself first: `What's the use case for this generic problem description?`. That leads to such considerations, didn't want to bother you, I appreciate your patience ;) ... – πάντα ῥεῖ Jul 24 '14 at 21:57
1

@πάνταῥεῖ: You *always* have exact values to compare. The values may or may not be what you want them to be depending on how they were formed and how hard you thought about the code. This is *very different* from the value of the number being noisy. – tmyklebu Jul 24 '14 at 22:18
@πάνταῥεῖ: "I tend to use some comparison involving `std::numeric_limits::epsilon`, to catch up such problems." This is bad advice. Every case has to be treated on its merits; using `std::numeric_limits::epsilon` by default is as bad as using `==` by default. – TonyK Jul 25 '14 at 20:45
@TonyK I explained very well, I don't choose this method as default, but use different patterns depending on the actual use case. – πάντα ῥεῖ Jul 25 '14 at 20:56
@πάνταῥεῖ: As far as I can see, you didn't explain anything at all. – TonyK Jul 25 '14 at 21:35
@TonyK Well, I can repeat it for you :P `"Either you don't care about small differences, then deal with floating point and epsilon, or even the smallest difference needs to be fixed up for your problem, then don't use floating point but fixed point values."` – πάντα ῥεῖ Jul 25 '14 at 21:39

James Curran · Answer 2 · 2014-07-25T19:46:37.280

0

First, we make state the standard caveats about float point numbers on computers. They are inherently imprecise. There's just no way to accurately say ".1" in binary. So, any of those may give you a "wrong" answer. But, in general, they should give you the same wrong answer.

In general, you'll have more variation with multiplication and division than with addition and substraction. And, you'll ofton run into trouble dealing with very large numbers and very small numbers in the same calculation.

[update] This next sentence is wrong. Ignore it:
However, as far as I can see, in this limit context, it seems everything balances out correctly, so while I can't say it categorically, I think you're safe here.

edited Jul 25 '14 at 19:46

answered Jul 24 '14 at 20:37

James Curran

101,701
37
181
258

You know that nobody's stopping you from either looking at the expressions and seeing what's up or from coding it up and trying it out, right? – tmyklebu Jul 24 '14 at 22:16
1

Trying something out doesn't guarantee that the result won't be different another time – M.M Jul 25 '14 at 00:08
@MattMcNabb: It's *very* hard to try these different expressions out for a bunch of values of `c` and `d` and not notice that they yield different results. – tmyklebu Jul 25 '14 at 05:42

score 0 · Answer 3 · answered Jul 25 '14 at 17:32

If I run the following code:

#include <stdio.h>

int main(int argc, char* argv[])
{
    float a = 0.1;
    float b = 0.2;
    float c = 0.3;
    float d = 0.4;

    float result1 = a + b + c + d - c;
    float result2 = a + b + c + (d - c);
    float result3 = a + b + d;

    printf("result1 == result2: %s\n", result1 == result2 ? "Yes" : "No");
    printf("result2 == result3: %s\n", result2 == result3 ? "Yes" : "No");
    printf("result1 == result3: %s\n\n", result1 == result3 ? "Yes" : "No");

    printf("result1: %30.20f\n", result1);
    printf("result2: %30.20f\n", result2);
    printf("result3: %30.20f\n\n", result3);

    printf("a + b:               %30.20f\n", a + b);
    printf("a + b + c:           %30.20f\n", a + b + c);
    printf("a + b + c + d:       %30.20f\n", a + b + c + d);
    printf("d - c:               %30.20f\n", d - c);
    printf("a + b + c + d - c:   %30.20f\n", a + b + c + d - c);
    printf("a + b + c + (d - c): %30.20f\n", a + b + c + (d - c));
    printf("a + b + d:           %30.20f\n", a + b + d);

    return 0;
}

I get the following output, showing that the answer to your main question is NO:

result1 == result2: No
result2 == result3: Yes
result1 == result3: No

result1:         0.69999998807907104492
result2:         0.70000004768371582031
result3:         0.70000004768371582031

a + b:                       0.30000001192092895508
a + b + c:                   0.60000002384185791016
a + b + c + d:               1.00000000000000000000
d - c:                       0.09999999403953552246
a + b + c + d - c:           0.69999998807907104492
a + b + c + (d - c):         0.70000004768371582031
a + b + d:                   0.70000004768371582031

Compiler: Apple LLVM 5.1 (clang).

I get the same results with one, but not with another compiler I tried with. The last one uses the x86 FPU in 32 bit code. The other two run in 64 bit mode and use SSE2, AFAIK.

Can we assume any pair of following floating point arithmetic statements always produce identical result?

3 Answers3