Stack Overflow
Stack Overflow

Questions tagged [floating-accuracy]

Concerning the accuracy of operations performed on floating point numbers.

Floating point numbers (typically meaning the IEEE standard) are inherently inexact and errors can compound, leading to edge cases in some decision processes, or numerical instability in certain algorithms.

Here is a mathematical treatment of the main problems.

1358 questions
-3
votes
2 answers

C++ static_cast Incorrect

I am new to C++ and have the following simple code snippet exploring C++ limitations: int main() { float x = 2147483000; int y = static_cast(x); cout << "x: " << x << ", y: " << y; } Why is the output showing me different values for x…
juco
  • 57
  • 6
-3
votes
2 answers

i need more decimal places for pi calculation

I'm trying to make a Pi calculator in python but I need more decimal places. it would help a lot if someone edited my code and carefully explained what they did. this is the code I'm using. import math d = 0 ans = 0 display = 0 while True: …
Not Gartificial
  • 1
  • 1
-3
votes
2 answers

Could we remove floating point errors for numbers if we changed the format to that similar for double IEEE 754-1985?

Currently IIRC, the current approach for displaying floating point numbers is to show them as 1/2 + 1/4 + 1/8 .... However what if we changed our approach to floating point numbers such that any floating point number is actually a normal integer,…
J.Doe
  • 31
  • 4
-3
votes
1 answer

Why does adding 1.0/3.0 numeric literal three times evaluates to exactly 1 in Golang?

The StackOverflow answer here is very complex for a beginner like me. Executing the following line of code in Golang results into 1 fmt.Println(1.0/3.0 + 1.0/3.0 + 1.0/3.0) According to my knowledge, 1.0/3.0 results into 0.3333333... which cannot…
Kathan Vakharia
  • 53
  • 7
-3
votes
2 answers

Subtracting floats in C language

#include #include float sum (float *A, int len) // sum of table of floats { float ss = 0.0; int i; for(i=0; i < len; ++i) ss +=A[i]; return ss; } void print(float A[][3], int row) //// procedure that…
pooo mn
  • 23
  • 1
  • 6
-3
votes
3 answers

Why is the float value changing in my C program?

In this program, the float array purchases[2] value is 90.50 but when I run the code it changes from 90.50 to 90.5010.990000. Why? purchases[0] and purchases[1] print fine. #include int main() { float purchases[3] = {10.99, 14.25,…
ziggelflex
  • 23
  • 4
-3
votes
2 answers

Double value appending with additional 0

Double amount = 0.001 but amount value is set to 0.0010 we have strict condition at the end to allow only 3 decimal places before storing in DB. but somehow it is appending 0 at the end. i have tried DecimalFormat decimalFormat = new…
Manjunath
  • 111
  • 1
  • 8
-3
votes
1 answer

Addition of floating point, Why the First code work

Why body of if() in 1st code Executes While the body of if() in 2nd code Doesn't Executes Float are not precise Working double num1 = 0.2; double num2 = 0.2; double num3 = num1 + num2; if (num3 == 0.4) { MessageBox.Show("1st"); } Not…
TeamKhurram
  • 107
  • 9
-3
votes
1 answer

weird result after calling a method that should return a floating number with exact number of decimals

I call the following method to convert the inputs as indicated in the body : public static double convertFromHectareAreCentiareToHectare(int hectare, int are, int centiare){ double d; DecimalFormat df = new…
mounaim
  • 1,132
  • 7
  • 29
  • 56
-3
votes
3 answers

Python representation of floating point numbers

I spent an hour today trying to figure out why return abs(val-desired) <= 0.1 was occasionally returning False, despite val and desired having an absolute difference of <=0.1. After some debugging, I found out that -13.2 + 13.3 =…
chazkii
  • 1,300
  • 12
  • 21
-3
votes
4 answers

My C++ code not showing floating point value

My code is not showing exact float value means number after point is not displayed by the code I am using Turbo C for sum up the series 1 + 1/3 + 1/5 + 1/7 + 1/9+….. Up to N terms #include #include void main() { int k=0; …
user2241865
  • 113
  • 4
  • 14
-3
votes
1 answer

Floating point inaccuracy in R?

Does R have a known problem with uniroot and handling floating points? >str(uniroot(function(x) x*(x^2-1) + .5, lower = -2, upper = 2, + tol = 0.0001)) List of 4 $ root : num -1.19 $ f.root : num -2.55e-07 $ iter : int…
gkb0986
  • 3,099
  • 1
  • 24
  • 22
-4
votes
1 answer

Multiplication of large numbers python

I decided to multiply two numbers: 1.0 17299991019999108999910899991119999 and as a result , I received in response: 1.729999101999911e+34 Code: print(1.0*17299991019999108999910899991119999) 1.729999101999911e+34 Here I counted it on a…
Kucer0043
  • 27
  • 7
-4
votes
2 answers

Why does converting an integer string to float and double produce different results?

let n = "77777777" let i = Int(n)! let f = Float(n)! let d = Double(n)! print(i) print(f) print(d) print(Int(f)) print(Int(d)) produces: 77777777 7.777778e+07 77777777.0 77777776 77777777 Why? Using Xcode 11, Swift 5. This question looks similar…
Abhijit Sarkar
  • 21,927
  • 20
  • 110
  • 219
-4
votes
2 answers

Does an number with integer value + hardcoded decimal number(same sign) generate same value as hardcoded expected decimal number (eg:10.0+0.1===10.1)?

Note: I'm not asking about Is floating point math broken? , because I asks about number with integer value+another decimal number with dec instead of decimal number+decimal number. for example, 10.0+0.1 generates a number with rounding errors, 10.1…
ocomfd
  • 4,010
  • 2
  • 10
  • 19
1 2 3
…
90
91