double addition result being rounded

Question

I have some code that is behaving strangely and seems to be rounding the result of the addition of two double values. This is causing issues with my code.

Unfortunately I cannot fix this issue because my unit testing environment is working OK (not rounding) and my application is not OK (rounding).

In my test environment:

a = -3.7468700408935547
b = 525218.0
c = b + a
c = 525214.25312995911

Inside my application:

a = -3.7468700408935547
b = 525218.0
c = b + a
c = 525214.25

What can be causing this? Project config? (I'm using visual studio, btw)

Edit (from comments)
I'm stepping through the same code using the Visual Studio debugger, so it's the exact same piece of code.

I have more code but I narrowed the problem down to that particular sum.

The binary representations of each value are:

test environment:

System.BitConverter.ToString(System.BitConverter.GetBytes(a))   "00-00-00-00-97-F9-0D-C0"   string
System.BitConverter.ToString(System.BitConverter.GetBytes(b))   "00-00-00-00-44-07-20-41"   string
System.BitConverter.ToString(System.BitConverter.GetBytes(c))   "00-40-9A-81-3C-07-20-41"   string

inside application:

System.BitConverter.ToString(System.BitConverter.GetBytes(a))   "00-00-00-00-97-F9-0D-C0"   string
System.BitConverter.ToString(System.BitConverter.GetBytes(b))   "00-00-00-00-44-07-20-41"   string
System.BitConverter.ToString(System.BitConverter.GetBytes(c))   "00-00-00-80-3C-07-20-41"   string

Edit 2:

As Alexei Levenkov points out, this issue is caused by a library that changes the FPU config.

_{For anyone who is curious what this meant for me:
I was able to mitigate this issue for my particular piece of code, by making some assumptions about my input values and doing some preemptive values rounding which in turn made my calculations consistent.}

How do you know the actual value is being rounded, and not just its display? — Jon B, Oct 30 '12 at 16:35
What are the variables defined as? It is likely `b + a` is getting rounded to 2DP due to variable choice... — MoonKnight, Oct 30 '12 at 16:35
Not every value can be accurately represented in binary, much like `1/3 = 0.333333333; 0.333333333 * = 0.99999999` in decimal, but then for different values. — Jasper, Oct 30 '12 at 16:37
@lboshuizen it's the same code being stepped, one at application runtime and other from a breakpoint in nunit. — João Portela, Oct 30 '12 at 16:57
@Killercam all the variables are doubles, at first I had some floats mixed with the doubles but then I changed the code to convert to doubles before doing anything else (just to be sure It wasn't the mix between floats and doubles). — João Portela, Oct 30 '12 at 17:01
Does you application do something strange with configuration of FPU? (like using some random library for math which reconfigures precision...) — Alexei Levenkov, Oct 30 '12 at 17:02
@AlexeiLevenkov now that you mention it, it might be the 3D engine I'm using. — João Portela, Oct 30 '12 at 17:04
@AlexeiLevenkov My 3D library uses Direct3D, which seems to be a [common cause for these issues](http://stackoverflow.com/questions/11524536/pow-implementation-for-double). Do you want to make your comment an answer? — João Portela, Oct 30 '12 at 17:14

ΩmegaMan · Answer 1 · 2012-10-30T17:38:27.237

1

Use decimal if one wants preciscion such as needed in financial calculations.

Edit See:

edited Oct 30 '12 at 17:38

answered Oct 30 '12 at 16:45

ΩmegaMan

29,542
12
100
122

I thought of that, but i really need the speed provided by these data types, the only problem is that I cannot understand the difference between the environments to make the code work at all times. – João Portela Oct 30 '12 at 16:49
@JoãoPortela the lack of precision is why doubles are used as such and that is the trade off you face. – ΩmegaMan Oct 30 '12 at 17:10
I am aware of that, but I can never be sure if my fix is correct if I don't know what the problem is in the first place. That's what this question was for: understanding the problem. – João Portela Oct 30 '12 at 17:26
I didn't know your second link. Thanks for that. – João Portela Oct 30 '12 at 17:44
Wow. I'm learning C#/VS and my background comes from PHP where data types is not a huge issue. It blows my mind that 'Double' is screwing with my data and I [have] to use 'Decimal'... – Aug 22 '14 at 02:09

score 1 · Accepted Answer · edited May 23 '17 at 11:48

1

Your application may be doing something strange with configuration of FPU. I.e. using some random library for math which reconfigures precision...

Direct3d is possible suspect, see for example Pow implementation for double.

edited May 23 '17 at 11:48

Community

1
1

answered Oct 30 '12 at 17:28

Alexei Levenkov

98,904
14
127
179

double addition result being rounded

2 Answers2