underlying data structure for float in python

Question

Got a question regarding to the underlying data structure of float (and precision) in Python:

>>> b = 1.4 + 2.3
>>> b
3.6999999999999997

>>> c = 3.7
>>> c
3.7000000000000002

>>> print b, c
3.7  3.7

>>> b == c
False

it seems the values of b and c are machine dependent, they are the numbers that closest to the target values but not exactly the same numbers. I was supervised that we get the 'right' numbers with 'Print', and someone told me that it was because print 'lies' while Python chose to tell us the truth i.e. showing exactly what they have stored.

And my questions are:

1. How to lie? e.g. in a function we take two values and return if they are the same, how I could have a best guess if the number of decimal(precision) is unknown? like b and c mentioned above? is there a well defined algorithm to do that? I was told that every language (C/C++) will have this kind of issue if we have floating point calculation involved, but how do they 'solve' this?

2. why we cannot just store the actual number instead of storing the closest number? is it a limitation or trading for efficiency?

many thanks John

Answer 1

For the answer to your first question, take a look at the following (slightly condensed) code from Python's source:

#define PREC_REPR       17
#define PREC_STR        12

void PyFloat_AsString(char *buf, PyFloatObject *v) {
    format_float(buf, 100, v, PREC_STR);
}

void PyFloat_AsReprString(char *buf, PyFloatObject *v) {
    format_float(buf, 100, v, PREC_REPR);
}

So basically, repr(float) will return a string formatted with 17 digits of precision, and str(float) will return a string with 12 digits of precision. As you might have guessed, print uses str() and entering the variable name in the interpreter uses repr(). With only 12 digits of precision, it looks like you get the "correct" answer, but that is just because what you expect and the actual value are the same up to 12 digits.

Here is a quick example of the difference:

>>> str(.1234567890123)
'0.123456789012'
>>> repr(.1234567890123)
'0.12345678901230001'

As for your second question, I suggest you read the following section of the Python tutorial: Floating Point Arithmetic: Issues and Limitations

It boils down to efficiency, less memory and quicker floating point operations when you are storing base 10 decimals in base 2 than any other representation, but you do need to deal with the imprecision.

As JBernardo pointed out in comments, this behavior is different in Python 2.7 and above, the following quote from the above tutorial link describes the difference (using 0.1 as an example):

In versions prior to Python 2.7 and Python 3.1, Python rounded this value to 17 significant digits, giving ‘0.10000000000000001’. In current versions, Python displays a value based on the shortest decimal fraction that rounds correctly back to the true binary value, resulting simply in ‘0.1’.

Answer 2

You should read the infamous paper:

What every computer scientist should know about floating-point arithmetic

Click on the link that says "CACHED" to download the paper in PDF format.

Answer 3

You get a different result in your calculation because the numbers 1.4 and 2.3 are not represented exactly either. When adding them, you also accumulate their precision limitations.

All floating point numbers have a limited precision, and because of the way that floating point numbers are usually represented internally (using base 2 rather than base 10), the limitations apply to numbers that we humans percieve to be easy to represent exactly.

The limited precision is rarely a problem for calculations, as the precision is still enough for most applications. When comparing floating point numbers on the other hand, the limited precision has to be considered.

This is usually done by subtracting the numbers, and checking if the difference is small enough compared to the numbers.

So, for exmample, if:

abs(b - c) < abs(b) / 1000000000000

then you could consider them equal. How many digits you want to consider depends on the precision of the floating point number, i.e. if you are using single or double precision numbers, and what calculations you have done to reach the numbers. As the precision limits accumulate with each calculation, you might need to lower the threshold for when they are considered equal.

When displaying a floating point number, it's rounded corrseponding to it's precision. If for example it's capable of representing 15 digits accurately, it could be rounded to 13 digits before being displayed.

Floating point numbers are intended for fast calculations. There are other data types, like Decimal, that can store a number exactly. Those are used for example for storing currency values.

Answer 4

Floating point numbers are imprecise; it's a facet of the representation method. There's a lot of back information about precisely why this is; suffice it to say that it's an issue on pretty much any platform that provides floating point numbers.

The best way to deal with the imprecision is to have a confidence interval; that is, comparison of two calculated floats for equivalency can be problematic because the representations can be off by a tiny amount, so the way to deal with this is to subtract the two of them, and make sure the difference is no more than a small quantity. Many libraries already have this sort of functionality built in for floats, but it's not particularly hard to implement yourself when in doubt.

Answer 5

This lecture is a pretty good insight to how the variables are stored in-memory and the professor includes an example that would give the unexpected results you are seeing.
http://www.youtube.com/watch?v=jTSvthW34GU If you need to compare the numbers cast them both as integers first and you will notice that they do equal if you perform the test.

Answer 6

All numbers are stored on a limited numbers of bits, hence you cannot just store the actual number and have to live with storing the closest number (imagine a fraction 1/3, if you want to store it on paper using decimal numbers, you will run out of world's recourses of trees). The alternative is symbolic representation you can find for example in Mathematica, which is just storing 1/3 as 1 and 3, but it's far away from machine and makes computations slower and more complicated.

Take a look at some links people are posting here and read about floating point numbers... it's a little bit scary though and you won't trust machines anymore.