Anything related to double-precision floating-point arithmetic and data-types. Often used with reference to IEEE754 double-precision floating-point representation.
Questions tagged [double-precision]
240 questions
2
votes
1 answer
Calculating Markov chain probabilities with values too large to exponentiate
I use the formula exp(X) as the rate for a markov chain. So the ratio of selecting one link over another is exp(X1)/exp(X2). My problem is that sometimes X is very large, so exp(X) will exceed the range of double.
Alternatively: Given an array of…
Masood_mj
- 1,144
- 12
- 25
2
votes
1 answer
Is it possible to format a double with a SimpleCursorAdapter?
I use the following SimpleCursorAdapter:
String campos[] = { "nome_prod", "codbar_prod",
"marca_prod", "formato_prod", "preco"};
int textviews[] = { R.id.textProdName, R.id.textProdCodBar, R.id.textProdMarca,
R.id.textProdFormato,…
jRicardo
- 804
- 10
- 20
2
votes
2 answers
C++ : sin(M_PI/6) = 0.5?
I am writing a scientific program which uses common values of sines in its main algorithm, namely sin(M_PI/N) for N = 1, 2, 3, 4, 5, 6.
Since I want my program to be as fast as possible, I thought : let's store these values in a vector instead of…
Seub
- 2,451
- 4
- 25
- 34
2
votes
2 answers
trying to convert double presision number to decimal
I'm trying to convert double precision number to decimal .
for example the number 22 in double precision and as bytes are :
[0] 0
[1] 0
[2] 0
[3] 0
[4] 0
[5] 0
[6] 54
[7] 64
now I try to convert these values again to 22 :
ByteBuffer buffer =…
melisa zand
- 211
- 2
- 6
- 16
2
votes
3 answers
c++ String is shortening double when printing
This string operation prints out a double in short-hand, and I can't work out why. Why is this happening, and how can I get the full output like the first line of output?
string myString = "The value is ";
ss.str(""); // stringstream from…
jwbensley
- 10,534
- 19
- 75
- 93
1
vote
1 answer
Double precision in image ratio comparison
I'm really weak with math and having a problem with an image resize algorithm.
I'm trying to resize an image to a a specific ratio.
double neededRatio = 1.6d;
while (!AboutEqual(imageRatio, neededRatio))
{
var cropPixels = 10;
//crop code
…
Madd0g
- 3,841
- 5
- 37
- 59
1
vote
4 answers
double granularity affecting numeric comparisons c++
Below is a test block of code that is supposed to determine equivalency of 2 doubles within a tolerance.
double lhs_1 = 0.02;
double lhs_2 = 0.04;
double rhs = 0.03;
double tolerance = 0.01;
bool is_match_1 = (abs(lhs_1 - rhs) <= tolerance);
bool…
Ian
- 4,169
- 3
- 37
- 62
1
vote
1 answer
Precision long double worse than double
When I use long double I get worse precision than using double.
3.14159265358979323846264L is this good to write long double const in source code or I should add something other than L?
EDIT
I solved the problem. I change algorithm to be more…
Bla bla
- 111
- 2
- 8
1
vote
4 answers
C++ double precision and rounding off
I have the following problem:
double a = 6.005; double b = 5.995;
I want to set precision of doubles 2 digits after point, for example
double c = a+b;// I would like to get 11.99 not 12.00.
How can I do this?
user712644
1
vote
3 answers
Java: Trigonometry and double inaccuracy causing NaN
I have a distance formula using latitude and longitude:
distance = EARTH_MILES_RADIUS
* Math.acos(Math.sin(lat1 / RADIAN_CONV)
* Math.sin(lat2 / RADIAN_CONV)
+ Math.cos(lat1 / RADIAN_CONV)
* Math.cos(lat2 / RADIAN_CONV)
*…
Muhd
- 24,305
- 22
- 61
- 78
1
vote
1 answer
Doubles returned and bound to list view are truncated
I am query my sqlite data base for my Android app.
I then use a simple cursor adapter to bind the data to a list view.
there are two doubles being called from the database and on the list view the only show up as 6 digits long.
Example: 43.6234 and…
Seth Hikari
- 2,711
- 6
- 27
- 32
1
vote
1 answer
Subtracting 2 different doubles yields zero
I'm building a loc-based app, using CLLocationManager and locationManager:didUpdateToLocation: fromLocation:
I'm trying to determine if the user has moved enough to trigger some other processing.
I'm using double variables. I learned long ago not…
Rayfleck
- 12,116
- 8
- 48
- 74
1
vote
1 answer
Matlab: Storing large numbers in Matlab
For storing large numbers I am refering to the next:
2.7182818284590455348848081484902650117874145507812500
I can´t save this with double precision floating-point format (IEEE754)
Thank you.
Peterstone
- 7,119
- 14
- 41
- 49
1
vote
0 answers
Double precision in compute shader (full code)
I send double precision data to my compute shader using an ssbo but somehow looses precision somewhere in the shader. I posted this problem here but has not yet received a response.
I read a position which contains 4 doubles that I use in…
Saalsa
- 11
- 3
1
vote
4 answers
Can I print -0.000 as 0.000?
printf("%.3lf\n", -0.0001);
This outputs -0.000, but shouldn't it be 0.000?
How can I make this print without the minus sign, i.e. 0.000 ?
Md Tahsin Rahman
- 124
- 1
- 8