Something is wrong with the accuracy of calculation between variables

Question

I have some problems with my code where I think the accuracy is a bit off. I'll take out the declarations of variables from my code, so the code is as small as possible:

int a = Int32.Parse(tb_weight.Text);
double b = 0;
b = (a * 1.03) / 1000;
double g = 0;
g = (1.09 + (0.41 * (Math.Sqrt(50 / b))));
lbl_vertforce.Content = Math.Round((b * g * 9.81), 2);

So, tb_weight is a textbox where the input is made, and lets say the input is 5000, the label lbl_vertforce is showing 119,61 and according to my calculator, it should show 119,74. What is wroing here?

I think that your calculator is wrong. I just did it my self and it is 119.61 — Yacoub Massad, Feb 19 '16 at 16:33

score 1 · Answer 1 · edited May 23 '17 at 12:15

1

Doubles are not 100% precise and can vary in the least common digits. If you want exact precision you need to use Decimal type which has a bigger memory foot print, but was designed to be very precise. Unfortunately Math.Sqrt is not overloaded for Decimal and only works on doubles. I have provide code I found in another posting discussing the subject of Decimal Square roots: Performing Math operations on decimal datatype in C#?

public void YourCodeModifiedForDecimal()
{
int a = Int32.Parse(tb_weight.Text);
decimal b = 0;
b = (a* 1.03m) / 1000m;
decimal g = 0;
g = (1.09m + (0.41m * (Sqrt(50m / b))));
lbl_vertforce.Content = Math.Round((b* g * 9.81m), 2);
}

public static decimal Sqrt(decimal x, decimal? guess = null)
{
    var ourGuess = guess.GetValueOrDefault(x / 2m);
    var result = x / ourGuess;
    var average = (ourGuess + result) / 2m;

    if (average == ourGuess) // This checks for the maximum precision possible with a decimal.
        return average;
    else
        return Sqrt(x, average);
}

edited May 23 '17 at 12:15

Community

1
1

answered Feb 19 '16 at 16:52

Kelly Barnard

1,131
6
8

While this is true with respect to floating point numbers and how some numbers cannot be represented correctly, the OP's problem is not that the output is wrong, it's that he calculated the expected answer incorrectly. The outputted answer (119.61) is correct - I've just checked it myself, as have others in the comments. – user1666620 Feb 19 '16 at 17:02
I tried this, and I still got 119.61. I've now tried 3 different calculators home, and everyone is giving me 119.74. 5.15 x 2.37 x 9.81 = 119.74 How come everyone else get 119.61? – Thomas Eikje Gjerde Feb 19 '16 at 17:05
I suppose you are right. I have dealt with this issue before in programs that calculate money so I figured I would explain why doubles are bad if you are looking precision. – Kelly Barnard Feb 19 '16 at 17:07
@ThomasEikjeGjerde ahhh that's what it is. `(1.09 + (0.41 * (Math.Sqrt(50 / 5.15)))` is not equal to `2.37` - it's `2.3675......`. You need to round to 2 decimal places. – user1666620 Feb 19 '16 at 17:07
My Calculator shows 119.61029556, something wrong with how you are solving this on your calculator or something wrong with your calculator. – Kelly Barnard Feb 19 '16 at 17:13

user1666620 · Accepted Answer · 2016-02-19T17:13:35.080

0

You need to round g to 2 decimal places to get 119.74 in the final calculation.

g = Math.Round(1.09 + (0.41 * (Math.Sqrt(50 / b))), 2);

edited Feb 19 '16 at 17:13

answered Feb 19 '16 at 17:09

user1666620

4,800
18
27

Something is wrong with the accuracy of calculation between variables

2 Answers2