Wrong result for division of two doubles in release build

Question

When I compile my application in Release mode I get incorrect division result of 40.0 / 5 = 7. In debug compilation it is correct, and result is 8

I tried to cast to double, from double, to int, without abs() etc, but no luck. I know this must be related to weirdness of floating point math on computers, but I have no idea what exactly. I also logged the values on console, via the qDebugs() below the code - everything looks okay, except initial steps.

//somewhere in code
   double tonnageToRecover = 0.5;//actually, its QDoubleSpinBox->value(), with 0.5 step set. Anyway, the value finally reduces to 0.5 every time
   double tonnagePerArmorPoint = 0.0125;//taken from .json
   int minimumArmorDelta = 5;//taken from .json
...
//palace where the calculations are preformed
    double armorPointsPerHalfTon = tonnageToRecover / tonnagePerArmorPoint;    
    int steps = abs(static_cast<int>(armorPointsPerHalfTon / minimumArmorDelta));
    qDebug() << "armorPointsPerHalfTon = " << armorPointsPerHalfTon;
    qDebug() << "tonnagePerArmorPoint = " << tonnagePerArmorPoint;
    qDebug() << "steps initial = " << steps;
    qDebug() << "minimumArmorDelta = " << minimumArmorDelta;

both 1st division parts are type double, tonnageToRecover = 0.5, tonnagePerArmorPoint = 0.0125, result is 40 which is OK minimumArmorDelta is int = 5

So why 40/5 isn't 8??

Compiler - MinGW 32 5.3.0, from Qt 5.11 pack

Screenshots: Release Debug

Unless you give us a [mcve], and mention the compiler and version, we have no way of verifying this. — P.W, Apr 17 '19 at 09:16
Are you sure it's actually 40.0? Not something slightly less? You know floating point is "weird", yeah? — Lightness Races in Orbit, Apr 17 '19 at 09:36
I'm sure as much as I can believe this output - http://i64.tinypic.com/1zmzdrb.png — MasterBLB, Apr 17 '19 at 09:47
Print your values with greater precision, and include `tonnageToRecover`. — molbdnilo, Apr 17 '19 at 09:48
After an hour has passed: How about a [Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve)? Why do you expect us to create one? — Sebastian Mach, Apr 17 '19 at 10:24
Your basic problem is truncating the answer. If real number arithmetic would result in 8, floating point arithmetic will give an answer very close to 8 in either direction. Truncating, rather than rounding, reduces an answer even a tiny bit less than 8 to 7. You need to work out changes to your logic to avoid truncation. — Patricia Shanahan, Apr 17 '19 at 13:06
Well, I could understand rounding flaws, but damn, these are round numbers - 40 and 5. Not 39.(9), not 5.(0)1, but 40 and 5, so why division is not equal to 8??. Well, unless qDebug() is lying to me, that's it. — MasterBLB, Apr 17 '19 at 14:05
@MasterBLB I don't know qDebug, but many debug utilities do lie, in the sense of rounding floating point numbers. A number very slightly less than 40 might be reported as 40. — Patricia Shanahan, Apr 17 '19 at 17:42

Julian · Answer 1 · 2019-04-18T11:18:49.993

I guess, the reason is that armorPointsPerHalfTon / minimumArmorDelta could be not 8 but actually 7.99999999 in the Release-version. This value then changes to 7 through the int-cast.

So, if the Debug version calculates armorPointsPerHalfTon / minimumArmorDelta = 8.0000001, the result is static_cast<int>(armorPointsPerHalfTon / minimumArmorDelta) = 8.

It's not surprising that Debug / Release yield different results (on the order of machine precision), as several optimizations occur in the Release version.

EDIT: If it suits your requirements, you could just use std::round to round your double to the nearest integer, rather than truncation decimals.

MasterBLB · Answer 2 · 2019-04-17T11:05:02.593

@Julian I suspect that too, but how can I overcome this obstacle? Will try to change steps to double, then cast to int again. RESUT: still does not work :/

I found a solution, but I have no idea exactly why it works now. Current code it:

    double armorPointsPerHalfTon = tonnageToRecover / tonnagePerArmorPoint;
//    int aPHT = (int)armorPointsPerHalfTon;
//    double minDelta = 5.0;//static_cast<double>(minimumArmorDelta);
    QString s(QString::number(abs(armorPointsPerHalfTon / minimumArmorDelta)));
    int steps = abs(armorPointsPerHalfTon / minimumArmorDelta);

#define myqDebug() qDebug() << fixed << qSetRealNumberPrecision(10)
    myqDebug() << "tonnageToRecover = " << tonnageToRecover;
    myqDebug() << "tonnagePerArmorPoint = " << tonnagePerArmorPoint;
    myqDebug() << "armorPointsPerHalfTon = " << armorPointsPerHalfTon;
    //myqDebug() << "aPHT = " << aPHT;//this was 39 in Release, 40 in Debug
    myqDebug() << "steps initial = " << steps;
    myqDebug() << "string version = " << s;
    myqDebug() << "minimumArmorDelta = " << minimumArmorDelta;// << ", minDelta = " << minDelta;
#undef myqDebug

I suppose that creation of that QString s flushes something, and that's why calculation of steps is correct now. String has incorrect value "7", though.

score 0 · Answer 3 · answered Apr 18 '19 at 09:18

Your basic problem is that you are truncating.

Suppose real number arithmetic would give an answer of exactly 8. Floating point arithmetic will give an answer that is very close to 8, but can differ from it in either direction due to rounding error. If the floating point answer is slightly greater than 8, truncating will change it to 8. If it is even slightly less than 8, truncating will change it to 7.

I suggest writing a new question on how to avoid the truncation, with discussion of why you are doing it.

Wrong result for division of two doubles in release build

3 Answers3