Stack Overflow
Stack Overflow

Questions tagged [floating-accuracy]

Concerning the accuracy of operations performed on floating point numbers.

Floating point numbers (typically meaning the IEEE standard) are inherently inexact and errors can compound, leading to edge cases in some decision processes, or numerical instability in certain algorithms.

Here is a mathematical treatment of the main problems.

1358 questions
0
votes
1 answer

Largest floating point variable

The program takes for parameter one double and its doing computation with long float point values. eg double myvar= n*0.000005478554 /298477. The problem is that im not sure that the real computational value is inserted to myvar. because whenvever i…
Finlay Lifny
  • 43
  • 1
  • 6
0
votes
1 answer

Floating Point division, floor, and when a simple C++ Change Computing program gets complex

Floating point division is causing a value that should be 1 to be .9999999 thus flooring it becomes 0 instead of 1 so the calculation for the required change is incorrect. What's the simplest way to deal with this? (Also if there's some sort of…
Bryce L
  • 5
  • 4
0
votes
2 answers

How does Simulink implement the Runge-Kutta Solver?

I have a question on the same topic in math.stackexchange.com: https://math.stackexchange.com/questions/664422/how-does-matlab-simulink-get-such-accurate-ode-solution-results, and it remains open. I am looking to understand how Simulink's accuracy…
J Collins
  • 2,106
  • 1
  • 23
  • 30
0
votes
3 answers

Why am I getting rounding errors when using Ruby Rational?

I'm calculating programatically the frequencies of given musical notes. Quick intro: The musical note A4 has a frequency of 440Hz Notes one octave higher are double the frequency of the same note in a lower octave (so A3 is 220Hz, A2 is 110Hz) The…
bodacious
  • 6,608
  • 9
  • 45
  • 74
0
votes
4 answers

Why don't I get zero when I subtract the same floating point number from itself in Perl?

Possible Duplicates: Why is floating point arithmetic in C# imprecise? Why does ghci say that 1.1 + 1.1 + 1.1 > 3.3 is True? #!/usr/bin/perl $l1 = "0+0.590580+0.583742+0.579787+0.564928+0.504538+0.459805+0.433273+0.384211+0.3035810"; $l2 =…
Prashant
  • 47
  • 1
  • 4
0
votes
0 answers

Working with money and CakePHP

I'm building a web application with CakePHP. This application will use lots of financial data, and perfect accuracy is required. As you may now, floats shouldn't be used when working with money/currency/financial data (Why not use Double or Float to…
marc casas
  • 133
  • 2
  • 9
0
votes
5 answers

Set accuracy of random numbers in java?

I was wondering if I am able to set the accuracy of the random double numbers that I generate. Random generator = new Random(); double randomIndex = generator.nextDouble()*10000; That produces the random numbers within 10000. How am I able to set…
jsh6303
  • 2,010
  • 3
  • 24
  • 50
0
votes
2 answers

Rounding error reduction?

Consider the following functions: #include #include #include #include template inline Type a(const Type dx, const Type a0, const Type z0, const Type b1) { return…
Vincent
  • 57,703
  • 61
  • 205
  • 388
0
votes
2 answers

How to determine the Jacobian when solving a nonlinear system by Newton-Raphson method

I am trying to solve some nonlinear systems by Newton's method and the solution accuracy is very important to my problem. without using the symbolic computation softwares, how can I compute the Jacobian of a general nonlinear system of polynomials…
LCFactorization
  • 1,652
  • 1
  • 25
  • 35
0
votes
1 answer

Ruby floating number subtraction wrong?

Can anyone explain why the following Ruby routine gives me a 2.0 as a result? I think rounding the floats is the root of the error. puts(999_999_999_999_999_9.0 - 999_999_999_999_999_8.0); = 2.0 The same error? puts(999_999_999_999_999_3.0 -…
medi_kirk
  • 1
0
votes
1 answer

Why there is a discrepancy in the result?

If I apply Binet Formula and Recursive formula for finding the fibonaci series, there is a discrepancy in result. Why? Basically I am a student and it's our assignment to implement the fibonacci series. So while making the experiment I came across…
priyanka.sarkar
  • 25,766
  • 43
  • 127
  • 173
0
votes
2 answers

Using decimal outside the finances

I always hear that floating point suffers from rounding errors and, therefore, decimal types must be used in place of the floating point for the finance. What do they mean by finance? Do they consider money alone whereas gold, copper, iron, honey,…
Val
  • 1
  • 8
  • 40
  • 64
0
votes
0 answers

how to deal with precision like a boss?

there, I'm drawing a contour (in x,z plan) with opengl, then I want to know if a point is inside or outside of the contour. so I get the xmin, xmax, ymin et ymaxvalues of that contour ( right now is just a circle) then I add -delta to each min…
The Unholy Metal Machine
  • 1,093
  • 2
  • 17
  • 36
0
votes
2 answers

Adding up fractions in PHP

I would like to create a loop that keeps adding a set fraction, here in my example 1/3, and which later I can check against for matches with integer values. Obviously when php adds 1/3 + 1/3 + 1/3 the result is 0.9999999, so i thought I could use…
EnglishAdam
  • 1,380
  • 1
  • 19
  • 42
0
votes
1 answer

Error handling to check if float is .02f

Hi in my settings for an app the user must input a number that has 2 decimal digits no more or no less. So values such as 125.22, 11.09, 63.88 are acceptable values such as 1.1, 23.1823, 12, 293.0001 are not. How can I make an if statement checking…
4GetFullOf
  • 1,738
  • 4
  • 22
  • 47
1 2 3
…
90 91