Stack Overflow
Stack Overflow

Questions tagged [floating-accuracy]

Concerning the accuracy of operations performed on floating point numbers.

Floating point numbers (typically meaning the IEEE standard) are inherently inexact and errors can compound, leading to edge cases in some decision processes, or numerical instability in certain algorithms.

Here is a mathematical treatment of the main problems.

1358 questions
-2
votes
1 answer

Unexpected Value for the Calculation of Catalan's Constant

Here's the function: arbFloat catalan(){ arbFloat output, n = 0, first, second; std::string preComp, postComp; do{ preComp = resizeArbtoString(first); first += (1.0 / pow(2.0, 4.0 * n)) * ( (- 1.0 / (2.0 *…
esote
  • 831
  • 12
  • 25
-2
votes
1 answer

Unexpected decimal on a subtraction

I'm using Python 3.5. I have a basic operation but I get a wrong result: def main(argv): cms=None damin=4.815 dbmax=4.858 it=(dbmax-damin)*1000 print('Damin {0} Dbmax {1} IT {2}'.format(damin, dbmax, dbmax-damin)) So the…
Athanor
  • 855
  • 1
  • 16
  • 34
-2
votes
3 answers

Java and floating point arithmetic

Having the code public static final float epsilon = 0.00000001f; public static final float a [] = { -180.0f, -180.0f + epsilon * 2, -epsilon * 2 } The a is initialized as follows: [-180.0, -180.0, -2.0E-8] Instead of…
Antonio
  • 756
  • 7
  • 26
-2
votes
1 answer

Calling text file

if i have text file that has three column say 1 2 1 3 1 1 2 3 1 and also have a matrix s = [0.3 …
Asmaa Daoud
  • 69
  • 2
  • 8
-2
votes
2 answers

Impossible to achieve certain float value

I am testing my code. I have boolean conditions with float numbers. I assign a float value to test my code: float testFloatValue = 0.9; And the boolean condition is not met: if (testFloatValue == 0.9) { } Because when I debugging, my float number…
David
  • 1
  • 2
-2
votes
1 answer

floating elements next to each other

I am making a website and i want somthing like so: img text img img text img img text img the images will be on the left and right side of the screen, but when i float an image to the right it is under the images on the left. Here is an example:…
John Smith
  • 101
  • 1
  • 1
  • 9
-2
votes
2 answers

different result of int multiplication vs float multiplication

I have this code: float number1 = 2f / 12; System.out.println(number1); int number2 = 6; float x = number1 * number2; System.out.println(x); And as output I get: 0.16666667 1.0 Why? When I multiply number1 and number2, number2…
The Nightmare
  • 701
  • 5
  • 16
  • 36
-2
votes
2 answers

Floating-point inaccuracy with math operations and integers

Hey so I'm trying to learn Java, and I know about floating-points not being accurate and what not, but I'm feeling blind at the moment. I know I'm doing something wrong, the computer is not wrong, but I don't know where. I'm casting the Math returns…
B. Lee
  • 191
  • 6
-2
votes
2 answers

how to eliminate any number after the floating point?

Is there any function returns only the real number without the floating point? For an example, func(1.xxx)->1 func(52.xx)->52 func(0.xx)->0 Is there any function does so?
Amrmsmb
  • 1
  • 27
  • 104
  • 226
-2
votes
1 answer

Someone tells me how the floating number 1.002 * 10 ^3 is represented in the computer?

The question is in the title. Say, for a 64-bit machine. I want to know how the floating number is represented because I would like to know the result of 1.002 * 10^3 - 1.000 * 10 ^3 in machine representation. Thanks for your ideas.
zell
  • 9,830
  • 10
  • 62
  • 115
-2
votes
1 answer

C++ float rounding (error?)

Why when I use multiplication float a= 1.0500000f; float b= a*100.0f; why b is 104.99999 but not 105.0 ? and when I int f= (int)b; f is 104
mrgloom
  • 20,061
  • 36
  • 171
  • 301
-2
votes
1 answer

Inconsistent value in floating point operations JAVA

I don't know whether this question is asked before. I couldn't find it in searches. System.out.println(1.00 - 9*.10); the above statement prints out 0.09999999999999998 instead of simply 0.01 I know this is a well known problem in JAVA but i want…
Manu Viswam
  • 1,606
  • 2
  • 18
  • 33
-2
votes
2 answers

Invalid value when using floats

The following operation give out the wrong result. $First = '45.4000'; $Second = '4.6800000000'; $Third = '50.00'; echo ( $First + $Second ) - $Third; OUTPUT: 0.079999999999998 Expected Output: 0.08 I am looking on how to get the…
Tareq
  • 92
  • 4
-2
votes
2 answers

simple arithmetic with floats is rounded to wrong value

In my program I have a running total of a particular number, declared as a float before the main so as to be universal, and on every iteration I add and subtract floats from it. These floats are always numbers between 0 and 10, to one decimal place.…
zephyr
  • 215
  • 2
  • 13
-2
votes
1 answer

C sqrt == long int

I don't know why if I have the following code: int main() { long int height_cat, number_worker_cats, number_helper_cats, height_tree; bool flag; scanf("%ld%ld", &height_cat, &number_worker_cats); for (number_helper_cats = 1; ;…
Roger PA
  • 17
  • 1
  • 3
1 2 3
…
90 91