Anything related to the precision of a floating-point number representation. The term precision refers to the number of significant digits a representation can hold. This is NOT the same as the "accuracy", which concerns errors in performing calculations, although it may be sometimes related.
Questions tagged [floating-point-precision]
551 questions
7
votes
3 answers
Java - maximum loss of precision in one double addition/subtraction
Is it possible to establish, even roughly, what the maximum precision loss would be when dealing with two double values in java (adding/subtracting)? Probably the worst case scenario is when two numbers cannot be represented exactly, and then an…
Bober02
- 15,034
- 31
- 92
- 178
7
votes
3 answers
Division by zero prevention: Checking the divisor's expression doesn't result in zero vs. checking the divisor isn't zero?
Is division by zero possible in the following case due to the floating point error in the subtraction?
float x, y, z;
...
if (y != 1.0)
z = x / (y - 1.0);
In other words, is the following any safer?
float divisor = y - 1.0;
if (divisor != 0.0)
…
Max
- 1,203
- 1
- 14
- 20
7
votes
1 answer
Any equivalent of "extended" for C#?
I'm working on a new version of my Mandelbrot screensaver and I'm running out of floating point accuracy - simple double values don't have enough significant figures for my needs.
More significant figures = greater levels of zooming into the…
Bevan
- 43,618
- 10
- 81
- 133
6
votes
5 answers
Floating-point comparison of constant assignment
When comparing doubles for equality, we need to give a tolerance level, because floating-point computation might introduce errors. For example:
double x;
double y;
x = f();
y = g();
if (fabs(x-y)
CuriousMind
- 15,168
- 20
- 82
- 120
6
votes
4 answers
How java stores float v double
I realize a decimal number can only be so precise stored as a float in a binary system but what I don't understand is what is happening at the 7 decimal place in my output of 10/7.
In my first output test I noticed that the 5 (7th place) at the end…
user1178761
- 61
- 1
- 2
6
votes
6 answers
underlying data structure for float in python
Got a question regarding to the underlying data structure of float (and precision) in Python:
>>> b = 1.4 + 2.3
>>> b
3.6999999999999997
>>> c = 3.7
>>> c
3.7000000000000002
>>> print b, c
3.7 3.7
>>> b == c
False
it seems the values of b and c…
John
- 2,107
- 3
- 22
- 39
6
votes
4 answers
Do floats, doubles, and long doubles have a guaranteed minimum precision?
From my previous question "Is floating point precision mutable or invariant?" I received a response which said,
C provides DBL_DIG, DBL_DECIMAL_DIG, and their float and long double
counterparts. DBL_DIG indicates the minimum relative decimal
…
Wandering Fool
- 2,170
- 3
- 18
- 48
6
votes
1 answer
Precision of repr(f), str(f), print(f) when f is float
If I run:
>>> import math
>>> print(math.pi)
3.141592653589793
Then pi is printed with 16 digits,
However, according to:
>>> import sys
>>> sys.float_info.dig
15
My precision is 15 digits.
So, should I rely on the last digit of that value (i.e.…
Diego Andres Alvarez Marin
- 175
- 11
6
votes
1 answer
C++: convert "boost::multiprecision::float128" to "double"
I'm using the boost multiprecision library, and more precisely the boost::multiprecision::float128 type. Using ICPC for compiling, I get some errors when trying to to do something like:
double a = functionA();
where functionA() return a…
user1403546
- 1,680
- 4
- 22
- 43
6
votes
1 answer
How to define custom float-point format (type) in C++?
How can I define my own float-point format (type) with specific precision and certain bitness of exponent and significand? For example, 128-bit float-point number with 20-bit exponent and 107-bit significand (not standart 15/112-bit), or 256-bit one…
Eugene
- 1,037
- 4
- 20
- 34
6
votes
1 answer
Are Floats' Bit Patterns Ordered?
Looking at the IEEE float/double representation
[mantissa sign][signed exponent][unsigned mantissa]
Am I correct to assume that sorting these values numerically always results in the same as sorting the bit patterns themselves lexicographically?
My…
Dexter
- 3,072
- 5
- 31
- 32
6
votes
3 answers
Full precision display of floating point numbers in C++?
I have read several topics about the display of floating point numbers display in C++ and I couldn't find a satisfying answer.
My question is: how to display all the significant digits of a floating point numbers in C++ in a scientific format…
Vincent
- 57,703
- 61
- 205
- 388
6
votes
4 answers
Are multiples of 0.25 exactly representable as double?
I have the following code for finding quartiles:
#include
#include
typedef struct {
double qrt[3];
double *value;
int count;
} t_data;
static void set_qrt(t_data *data, int qrt)
{
int n, e;
double d;
d…
David Ranieri
- 39,972
- 7
- 52
- 94
6
votes
5 answers
Any risk of using float variables as loop counters and their fractional increment/decrement for non "==" conditions?
Are we safe to use floats as loop-counters and to increment/decrement them by fractional amounts at each iteration,like in the seemingly risk-free program below?Of course I know that using floats as operands for the == operator is a dumb thing to…
Rüppell's Vulture
- 3,583
- 7
- 35
- 49
6
votes
5 answers
Error With Using (int) and (double) together to Cut off Decimals
When I am using (int) with (double) some times it is not working correct.
Look At The PHP Code Example:
I Need To LEAVE 2 Decimals And REMOVE Other...
I Know number_format(); function But I Cannot Use It. Because It Is Rounding…
VoW
- 315
- 2
- 13