Stack Overflow
Stack Overflow

Questions tagged [double-precision]

Anything related to double-precision floating-point arithmetic and data-types. Often used with reference to IEEE754 double-precision floating-point representation.

240 questions
3
votes
1 answer

Precise division of doubles representing integers exactly (when they are divisible)

Given that 8-byte doubles can represent all 4-byte ints precisely, I'm wondering whether dividing a double A storing an int, by a double B storing an int (such that the integer B divides A) will always give the exact double corresponding to the…
Evgeni Sergeev
  • 22,495
  • 17
  • 107
  • 124
3
votes
2 answers

How to compare double numbers?

I know that when I would like to check if double == double I should write: bool AreSame(double a, double b) { return fabs(a - b) < EPSILON; } But what when I would like to check if a > b or b > a ?
yak
  • 3,770
  • 19
  • 60
  • 111
3
votes
3 answers

Function return type mismatch

I'm attempting to recode an old C++ program in Fortran to make use of LAPACK (I'm aware that C++ does have LAPACK++, but I'm having a lot of trouble installing it, so I gave up). I originally didn't have any problems with compilation, but that was…
Dave Williams
  • 376
  • 1
  • 3
  • 13
3
votes
2 answers

string.format(format,doubleValue) , precision lost

I have this double value: var value = 52.30298270000003 and when I convert it to string, it losses its precision: var str = string.Format("{0} some text...", value); Console.WriteLine(str); // output: 52.3029827 The number of precision on my…
amiry jd
  • 27,021
  • 30
  • 116
  • 215
3
votes
3 answers

Switching between float and double precision at compile time

Where should I look at if I want to switch between float and double precision at compile time. Its like, if user wants everything in float instead of double precision how I can maintain this flexibility? In other words, how should I define a…
Shibli
  • 5,879
  • 13
  • 62
  • 126
3
votes
1 answer

CUFFT with double precision

I am experiencing some problems with CUDAs FFT library. I declared the inputs as cuDoubleComplex, but the compiler returns the error that this type is incompatible with parameters of type cufftComplex. After some search through the Internet, I found…
Pippo
  • 1,543
  • 3
  • 21
  • 40
3
votes
0 answers

quadmath in gdb

I am debugging with gdb and I find that the quadruple precision numbers are not displayed properly. The code I am testing with is: #include #include #include extern "C" { #include "quadmath.h" } int main () { char…
Matyas
  • 644
  • 6
  • 15
3
votes
2 answers

Double precision problems when zooming in

I have this algorithm that calcuates the mandelbrot value of a point (x0,y0) (x0 and y0 are somewhere between -1 and 1 i thought, not very important). This is all going very well when scale isn't getting too big, but at higher values of scale, the…
JimmyMcHoover
  • 764
  • 7
  • 15
3
votes
3 answers

What is the maximum precision for double data type in C?

To compare two double type variables in C, I have defined #define EQUALITY_EPSILON = 1e-8. I am doing the comparison as follows: if((img_score[i] - img_score[j]) >= EQUALITY_EPSILON){ // handle for ith score greater than jth score } else…
stressed_geek
  • 2,118
  • 8
  • 33
  • 45
3
votes
1 answer

Precision for Double.parseDouble() and String.valueOf()

Does the following statement holds for any double (Java primitive double precision IEEE-754) except NaN: Double.parseDouble(String.valueOf(d)) == d Said otherwise, does parsing a serialized (using String.valueOf()) double value always yields the…
gawi
  • 13,940
  • 7
  • 42
  • 78
2
votes
2 answers

matlab and c++ precision

Hi I'm rewriting a script from MATLAB to C++ using armadillo library for linear algebra and matrix. for getting more or less the same output i called cout method: cout.precision(4); cout.setf(ios::fixed); but i'm getting different result: Matlab…
nkint
  • 11,513
  • 31
  • 103
  • 174
2
votes
3 answers

how to fix double precision issue in java

Possible Duplicate: Java floats and doubles, how to avoid that 0.0 + 0.1 + … + 0.1 == 0.9000001? How can I overcome the precision issue with double multiplication in java android??Please note that I am converting a string value into double…
prajul
  • 1,216
  • 2
  • 15
  • 27
2
votes
3 answers

How accurate/precise is java.lang.Math.pow(x, n) for large n?

I would like to calculate (1.0-p)^n where p is a double between 0 and 1 (often very close to 0) and n is a positive integer that might be on the order of hundreds or thousands (perhaps larger; I'm not sure yet). If possible I would love to just use…
Michael McGowan
  • 6,528
  • 8
  • 42
  • 70
2
votes
2 answers

TimeSpan and double rounding errors

I'm dealing with physical entities, such as time, speed and distance, and performing simple math, such as distance = time * speed, etc. Speed and Distance are double values rounded to 8th digit, and for Time values a .NET TimeSpan is used. Since…
Borka
  • 2,169
  • 3
  • 20
  • 26
2
votes
3 answers

What's the most precision you might lose if you use double to convert a currency value to another currency and back?

I know the problem of doubles due to base 2 calculations: var total: Double = 0.1 total *= 0.1 total /= 0.1 println(total) // 0.10000000000000002 I understand that BigDecimal should be used for money related calculations, but in my case the little…
user1819676
  • 369
  • 1
  • 12
1 2 3
…
15 16