Stack Overflow
Stack Overflow

Questions tagged [floating-accuracy]

Concerning the accuracy of operations performed on floating point numbers.

Floating point numbers (typically meaning the IEEE standard) are inherently inexact and errors can compound, leading to edge cases in some decision processes, or numerical instability in certain algorithms.

Here is a mathematical treatment of the main problems.

1358 questions
0
votes
2 answers

Why float value is rounded in python?

I have read that minimal float value that Python support is something in power -308. Exact number doesn't matter because: >>> -1.42108547152e-14 + 360.0 == 360.0 True How is that? I have CPython 2.7.3 on Windows. It cause me errors. My problem will…
skaurus
  • 1,581
  • 17
  • 27
0
votes
2 answers

adjusting floats to maintain high precision

I am writing some Python code that requires a very high degree of precision. I started to use Numpy float64, but that didn't work as required, and I then started using the "Decimal" module, which then worked fine. I would ideally prefer, however, to…
user1551817
  • 6,693
  • 22
  • 72
  • 109
0
votes
0 answers

Floating point precision problems when comparing to a RGN

Possible Duplicate: Floating point comparison functions for C# I've read about all the problems with floating point precision and from what I've read the answer is in rounding. In my situation I have a double that has a value of .07 which…
Gary
  • 1,784
  • 4
  • 18
  • 38
0
votes
5 answers

Increasing float value

Possible Duplicate: Floating point inaccuracy examples I have the following line inside a WHILE loop, in C/C++: while(...) { x = x + float(0.1); // x is a float type. Is the cast necessary? } x starts as 0. The thing is, after my first loop,…
iL_Marto
  • 3
  • 1
  • 6
0
votes
2 answers

conversion bigq to mpfr with Rmpfr package

The help documentation of the Rmpfr R package claims that the .bigq2mpfr() function uses the minimal precision necessary for correct representation when the precB argument is NULL : Description: Coerce from and to big integers (‘bigz’) and…
Stéphane Laurent
  • 75,186
  • 15
  • 119
  • 225
0
votes
1 answer

Exponentiation of a large floating point imaginary argument

How do I go about calculating exp(i * x) where |x| >> 2 pi? I know that it can be done with an arbitrary precision library, but is there an algorithm to do it stably (without roundoff errors) that doesn't require a library?
Andrew Spott
  • 3,457
  • 8
  • 33
  • 59
0
votes
2 answers

When will I encounter issues with the type Double in C# with 8 decimal places for Bitcoin?

I remember reading issues about certain math operations and the type double, but I forget when they would occur, or how I need to deal with them. A "Bitcoin" is a float that has 8 decimal places. I'm assuming that I use they type double with it,…
makerofthings7
  • 60,103
  • 53
  • 215
  • 448
0
votes
2 answers

Converting pixels to dpi for mdpi and hdpi screens

I am using this code that I have found on another thread which is working fine on mdpi screens: public static float convertDpToPixel(float dp,Context context){ Resources resources = context.getResources(); DisplayMetrics metrics =…
Jonno
  • 1,542
  • 3
  • 22
  • 39
0
votes
3 answers

visual studio 2010 showing weird behavior on float/double increment

Possible Duplicate: Dealing with accuracy problems in floating-point numbers I am writing an OpenGl animation and am using a float variable "time" to keep track of the time. I am incrementing the "time" variable by 0.01 .I have certain conditions…
code4fun
  • 2,661
  • 9
  • 25
  • 39
0
votes
1 answer

VB.NET running sum in nested loop inside Parallel.for Synclock loses information

Below is the best representation I have been able to develop for calculating a running sum inside a loop that's nested inside a Parallel.for loop in VB.NET (Visual Studio 2010, .NET Framework 4). Note that when showing the results in `sum' to the…
user1493382
0
votes
1 answer

Multiply before normalization

Let vector v close to zero like {7.000e-07, 5.000e-06, 4.000e-07}. Is it good idea to multiply by, say, 1024, before normalization? Seems like such close values work good on PC where double is 80 bit internally and bad on ARM FPU.
demi
  • 5,384
  • 6
  • 37
  • 57
0
votes
2 answers

C: Adding Exponentials

What I thought was a trivial addition in standard C code compiled by GCC has confused me somewhat. If I have a double called A and also a double called B, and A = a very small exponential say 1e-20 and B is a larger value for example 1e-5 - why does…
Protoplanet
  • 49
  • 5
0
votes
2 answers

How to convert floating point input to integers and preserve maximum precision?

I have to use an algorithm which expects a matrix of integers as input. The input I have is real valued, therefore I want to convert the input it to integer before passing it to the algorithm. I though of scaling the input by a large constant and…
stressed_geek
  • 2,118
  • 8
  • 33
  • 45
0
votes
1 answer

Unexpected floating point comparisons

Consider the following code: #include #define k 0.7 #define p 0.5 int main() { float a=k,b=p; double aa=k; if(a
sasidhar
  • 7,523
  • 15
  • 49
  • 75
0
votes
2 answers

issue with float increment in Android

I am trying to increment value of float by 0.1 every time. But i am getting very strange result some time. float kmVal = 0.0f; EditText kms = (EditText) findViewById(R.id.km); kmVal = Float.valueOf(kms.getText().toString()); On Click of button…
Scorpion
  • 6,831
  • 16
  • 75
  • 123
1 2 3
…
90 91