Anything related to the precision of a floating-point number representation. The term precision refers to the number of significant digits a representation can hold. This is NOT the same as the "accuracy", which concerns errors in performing calculations, although it may be sometimes related.
Questions tagged [floating-point-precision]
551 questions
0
votes
0 answers
C++ Increasing precision above that of a double
Possible Duplicate:
Is there a C++ equivalent to Java's BigDecimal?
I recently posted about a C++ Pi approximating program I was making. You can find the code for that here: C++ Pi approximation program but the code itself isn't really relevant.…
user1546083
- 1,777
- 3
- 13
- 12
0
votes
1 answer
epsilon for various float values
There is FLT_MIN constant that is nearest to zero. How to get nearest to some number value?
As an example:
float nearest_to_1000 = 1000.0f + epsilon;
// epsilon must be the smallest value satisfying condition:
// nearest_to_1000 > 1000.0f
I would…
brigadir
- 6,874
- 6
- 46
- 81
0
votes
2 answers
roundToDecimals returns too many digits
I have a process which returns a floating point number. Then I use the roundToDecimals method to eliminate some digits.
roundToDecimals(weight,3)
----update june 22 2012 : 1.42 am----
public static double roundToDecimals(double d, int c) {
int…
Herman Plani Ginting
- 61
- 2
- 8
0
votes
1 answer
Calculate machine precision on gmp arbitrary precision
I'm trying to obtain the machine precision on gmp variables.
To that end, I adapted the code from wikipedia, to compute the precision of a gmp with a fixed precision:
int main( int argc, char **argv )
{
long int precision =…
Jorge Leitao
- 19,085
- 19
- 85
- 121
0
votes
1 answer
fractional binary subtraction
I am having difficulty understanding why the following binary subtraction gives the result that it does. I keep getting a different answer. I am trying to compute 0.1-x such that x is 0.00011001100110011001100. The answer should be…
-1
votes
1 answer
How do you use double to get the expected result
I was trying out subtracting numbers in java, and this gives me unexpected result
public class FloatWeird
{
public static void main(String[] args)
{
double n = 1;
for(int i = 0;i<10;i++)
{
…
saltandwater
- 791
- 2
- 9
- 25
-1
votes
2 answers
IEEE754 float point substraction precision lost
Here is the subtraction
First number
Decimal 3.0000002
Hexadecimal 0x4040001
Binary: Sign[0], Exponent[1000_0000], Mantissa[100_0000_0000_0000_0000_0001]
substract second number:
Decimal 3.000000
Hexadecimal 0x4040000
Binary: Sign[0],…
Shuaiyu Jiang
- 239
- 1
- 3
- 15
-1
votes
1 answer
Convert a float to an int
I have a price for a car, let's say 10000. I want to apply a 20% sale to this price.
I have a struct in which auta->cena is a float.
int year, i, n=0, pocet=1, sale;
scanf(" %d", &year);
scanf(" %d", &sale);
for(i=1; i<=pocet; i++){
if(year ==…
domestos
- 7
- 4
-1
votes
3 answers
How do I filter out floats with integers [logic]
I have an array of floats. Some of these floats are VERY close to integers, but they're still floats. I need to filter these values.
For example:
array_n = [n]; //an array of floats
k = 160.
In the if loop, n is an element in array_n. I need to…
user3552360
- 1
- 1
-1
votes
4 answers
is_float not detecting float numbers
I am using is_float() to check if the number entered is float or not but it keeps throwing the echo() message when I enter 3.00. Is there any other solution to it? Or am I doing anything wrong?
Here is my code:
PHP:
if(!is_float($_POST["gpa"])){
…
user4244510
- 79
- 1
- 8
-1
votes
1 answer
ORA-00907: missing right parenthesis, FLOAT datatype issue, Oracle 11gR2
I have been trying to store the price of the products. So I need 2 decimal points and datatype should be float, i think.
I am using Oracle 11gR2. Following is what I have written.
create table test (
product varchar2(20),
price float(5,2));
It…
Murshed
- 3
- 2
- 2
- 7
-1
votes
1 answer
floating point precision using memcpy C++
I have a tricky problem that I do not understand. I have an array of uint8_t that I need to convert to a 32 bit floating point. I am using memcpy to accomplish that...however the rounding seems to be off. Could someone please explain what is…
user2494298
- 299
- 3
- 7
- 23
-1
votes
3 answers
how to handle result of cos(M_PI/2)
M_PI is a macro which is defined as 3.14159265358979323846, which seems more precise than a double. I'm trying to get the answer in a float and no amount of casting will help me, the result is always 6.12323426e-017 or -4.37113883e-008 if i try and…
cool mr croc
- 725
- 1
- 13
- 33
-1
votes
4 answers
How to ensure that my double output is ONLY 2 decimal places?
I am working on a class project that is suppose to be a ATM machine. The parameters for the project are to have 5 methods and a main program. Everything seems to be working fine, but upon further testing, and depositing more money, the new balance…
mlopman
- 67
- 1
- 1
- 7
-1
votes
1 answer
C++ Builder and Double Precision
I am trying to add double numbers which begin with 0.5,0.6... to 179.9,180.0.
To do it, I used the following code;
void __fastcall TForm1::FormCreate(TObject *Sender)
{
for (double i = 1; i <= 180; i+=0.1) {
listDegrees->Items->Add(i);
…
user3552878
- 9
- 2