Anything related to double-precision floating-point arithmetic and data-types. Often used with reference to IEEE754 double-precision floating-point representation.
Questions tagged [double-precision]
240 questions
0
votes
2 answers
How to convert sql type double-precision in c#
I have a database access, and one of this field is double-precision.
Normally if i set in the textbox 1.71 or 1,71 the field in database should contain 1.71.
But if i execute query the value of acces'field is 171 !!.
public const string QUERY =…
puti26
- 431
- 3
- 14
- 31
0
votes
1 answer
How does C store a double into an allocated array of type float?
I am changing my program from floating-point precision to double precision.
Nevertheless I start this routine by reading floating point numbers from a binary file.
bufz = alloc1float(nz);
for (i=0; i
MichaelScott
- 387
- 1
- 3
- 21
0
votes
1 answer
Adding check constraint for type double precision in postgres
I have coordinates with the type double precision in a column. I want to check the length (it has to be 7) before the decimal point (before the decimal places). The numbers can have decimals up to 4.
My first idea:
CHECK…
Stefan
- 1,383
- 2
- 15
- 25
0
votes
2 answers
How to Map a double value to a float one In Java?
At first glance, my question could seem a bit odd; but casting a double to a float value is not what i want. Since when you cast it, you just lose some precision with respect to the rules defined IEEE-754 and can't achieve actual mapping of a double…
iliTheFallen
- 466
- 7
- 16
0
votes
2 answers
Understanding double precision operations in C
I would like to understand why this code:
double r,d,rc;
scanf("%lf %lf", &r, &d);
rc = (r * r) - (d/2) * (d/2);
printf("%.2f\n", M_PI * rc);
returns more precise result than this one (without rc variable assignment):
double r,d,rc;
scanf("%lf…
syntagma
- 23,346
- 16
- 78
- 134
0
votes
3 answers
Explain 1 - 3*(4/3 - 1) = 2.2204e-16 in Matlab
I am trying to understand the double precision numbers in Matlab. Why is this 1 - 3*(4/3 - 1) not equal to zero?
hhh
- 50,788
- 62
- 179
- 282
0
votes
1 answer
How to translate the value of %a given by printf?
What is the value of 0x1.921fb82c2bd7fp+1 in a human readable presentation?
I got this value by printf using %a.
0x90
- 39,472
- 36
- 165
- 245
0
votes
2 answers
Matlab and c++ matrix product: same input, different outputs
I'm using Matlab 2010 in my ubuntu 12.04 on a x86 computer, and g++ 4.6.3.
this is how I do the production and the inputs:
#include
#include
using namespace std;
int main()
{
…
Mohammad
- 464
- 1
- 3
- 17
0
votes
1 answer
Double decimal places for coordinates
I am trying to convert a string to a double value to use it as a coordinate in an annotation.
I need the double value to have 5 decimal places to be placed on the map.
Let's say the string is hello and the double I need is whatsup.
The format of the…
B-Man
- 2,209
- 3
- 22
- 35
0
votes
1 answer
java: how can I convert double precision numbers to decimal
I want to convert double precision numbers to decimal .
for example I have 23.43, the double value would be (using matlab) : 174 71 255 122 20 110 55 64
now having this how can I write a java program that gives me 23.43 ?
and here is the…
melisa zand
- 211
- 2
- 6
- 16
0
votes
2 answers
Radeon HD 4850 and OpenCL: will cl_khr_fp64 work on this videocard?
This videocard (Radeon HD 4850) conforms only with OpenCL 1.0, per AMD Compatibility table. I need some hardware to conduct intensive financial calculations with doubleN types (no floats at all!). According to this cardtable, this card is able to…
toughbummer
- 1
- 1
0
votes
2 answers
Double precision - MS visual C++ 2005 Vs Matlab
I am using MS visual C++ 2005 for implementing some mathematical algorithms in C++. My C++ code takes 10K double data type input values with precision of 12 decimal places(For e.g. 866.333333333333) which are generated in Matlab. Then my code does…
Yogi
- 83
- 1
- 10
-1
votes
6 answers
Java Doubles are not good at math
I am currently writing a calculator program in java. It is my first java program, I am used to c++.
I have noticed that doubles in java are not at all like doubles in c++.
try this in java and c++
4.1*3
that/.1
it should be
12.3 then 123, and c++…
The Dude
- 310
- 7
- 15
-1
votes
1 answer
c# double multiply gets additional precision
after multiplying 2 double, it gets so many additional precision. Then it causes rounding(to 2 decimal) issue. Where I'm suppose to get 37.34, but it gives 37.33 instead. (viewing in debug mode)
additional precision…
Atlantiz8
- 65
- 1
- 2
- 11
-1
votes
1 answer
How do you use double to get the expected result
I was trying out subtracting numbers in java, and this gives me unexpected result
public class FloatWeird
{
public static void main(String[] args)
{
double n = 1;
for(int i = 0;i<10;i++)
{
…
saltandwater
- 791
- 2
- 9
- 25