Stack Overflow
Stack Overflow

Questions tagged [floating-accuracy]

Concerning the accuracy of operations performed on floating point numbers.

Floating point numbers (typically meaning the IEEE standard) are inherently inexact and errors can compound, leading to edge cases in some decision processes, or numerical instability in certain algorithms.

Here is a mathematical treatment of the main problems.

1358 questions
-1
votes
2 answers

Floating point numbers in PHP -- strange behaviour

Can someone explain what is happening here? I have floating point number that is to be rounded to two decimal points (price). echo $total . " is rounded to " . round((float)$total,2); var_export((float)$total); echo " is rounded to…
user1442439
  • 34
  • 6
-1
votes
1 answer

Best way to calculate machine epsilon in PHP?

What is best / most correct way of directly calculating machine epsilon (floating point rounding error) in PHP, not using built-in constant PHP_FLOAT_EPSILON ? Currently, I've managed to research two ways, "standard" and asymptotically approaching…
Agnius Vasiliauskas
  • 10,935
  • 5
  • 50
  • 70
-1
votes
4 answers

Number type with accuracy information?

Recently someone wanted a collision-free hash function to hash a million values to a 32-bit hash value. If you know the birthday paradox, you know that's unlikely to be collision-free. But wanting to know the probability, I computed it like this…
superb rain
  • 5,300
  • 2
  • 11
  • 25
-1
votes
1 answer

How such strange float format appears, both with dot and comma?

Recently I was refactoring tests in my application and broke one of them. But I can reproduce problem only if I run entire set of tests. If I run only one class with problem test, all works fine. Problem row: $float_value =…
Hayate
  • 653
  • 1
  • 9
  • 25
-1
votes
2 answers

Values not matching exactly when converting to float or decimal in python

When converting to float or decimal, the values are not correct. Below are some examples: "{0:.20f}".format(0.1) = '0.10000000000000000555' "{0:.20f}".format(1/3) = '0.33333333333333331483' Decimal(2.4) =…
Gaurav Kumar
  • 33
  • 5
-1
votes
1 answer

inaccurate results for calculations using floats - Simple solution

Many questions have been asked on StackOverflow and elsewhere about Python's confusing behaviour with calculations which use floats - often returning a result which is clearly wrong by a small amount. The explanation for this is invariably linked…
gnoodle
  • 149
  • 9
-1
votes
1 answer

How can I easily handle floating point inaccuracies in python?

I am generating an array, which contains very similar but not equal values, due to float inaccuracies. Here are sample elements from the beginning of the array: [1.6666666666651508, 1.6666666666651508, -0.9999999999990905, -0.6666666666660604,…
Lavair
  • 888
  • 10
  • 21
-1
votes
1 answer

Higher accuracy for tanh function in Tensorflow as compare to numpy

I can't figure out why tanh calculation has higher accuracy in tensorflow-2.2.0 as compare to numpy. Please have a look at example below. I appreciate your time and reply. Thanks,
Robin Sharma
  • 181
  • 2
  • 11
-1
votes
1 answer

Wolfram alpha and floating point arithmetic (loss of significance)

I'm studying floating point aritmetic. Suppose we are in double precision. We know that when we subtract two numbers which has "almost" the same magnitude, the relative error is large. In MatLab command window, for instance, if I…
lukk
  • 11
  • 2
-1
votes
3 answers

How to make more precise the reading of AnalogPins in Arduino?

I'm new here so, if I make any mistake, sorry. Well, I'm working with Arduino (Mega2560) to construct an Ammeter and found out a little problem... Arduino Mega measures voltage from 0 to 5V, and the AnalogPins return a 10-bit value according with…
Luan Souza
  • 159
  • 1
  • 1
  • 11
-1
votes
3 answers

Does exist two numbers that multiplied (or divided) each other introduce error?

Here's the bank of tests I'm doing, learning how FP basic ops (+, -, *, /) would introduce errors: #include #include int main() { std::cout.precision(100); double a =…
markzzz
  • 47,390
  • 120
  • 299
  • 507
-1
votes
1 answer

Why does .NET Core Convert.ToInt32 round up odd numbers

I have just been reading through some .NET Core code and tried to understand how System.Convert.ToInt32(float n) does its…
Michael Kargl
  • 662
  • 10
  • 21
-1
votes
3 answers

.NET trouble when using doubles

In .NET, when I subtract 1.35 from 1.35072 it shows .000719999999. How could I get .00072 when using a double? TOTKILO.Text = KILO.Text * TOUCH.Text * 0.01; //here 1.35072 TextBox10.Text = TextBox9.Text * TextBox8.Text * 0.01; //here 1.35 K =…
mukesh
  • 9
  • 2
-1
votes
1 answer

Low Accuracy in Implementing naive Bayes classifier

I have code for naive Bayes classifier that implement the concept of the naive Bayes, but the accuracy that this algorithm gives me is about 48% and it much lower than MATLAB build-in function for Naive Bayes (84%). Can anybody help me where is the…
Elnaz
  • 3
  • 1
-1
votes
2 answers

What can I do against floating point notations Errors?

I need to multiply a float by 100 in order to convert from € to cents. The problem I am having is, that the value of big numbers isn't calculated correctly. For example: String aa = "1000009.00"; aa = aa.replaceAll(",", "."); float bb =…
MrGatzi
  • 23
  • 3
1 2 3
…
90 91