3

I'm reading a JSON data file that might give me a float value of, say, 1.1. When I make a Decimal of that value, I get a crazy long number, because of the imprecision of binary representations of floats.

I understand binary representation and I'm okay with the idea that numbers writable in base-ten floating point can't always be represented in base-two.

But it seems that if I string-ify the float first, and make a Decimal using that string as the value, I get a Decimal without the tiny binary-imprecision delta.

Here's what I mean:

Python 2.7.6 (default, Jan 16 2014, 10:55:32) 
>>> from decimal import Decimal
>>> f = 1.1
>>> d = Decimal(f)
>>> f
1.1
>>> d
Decimal('1.100000000000000088817841970012523233890533447265625')
>>> d = Decimal(str(f))
>>> d
Decimal('1.1')

String-ifying the float before making it into a Decimal seems to give me a result closer to the original base-ten number as typed (or as read in from a JSON file).

So here's my question(s): when stringifying the float, why don't I see the long tail of digits? Is python automagically keeping track of the original string parsed in from the JSON, or something? Why doesn't the Decimal constructor use that trick too?

Daniel Baird
  • 2,239
  • 1
  • 18
  • 24
  • 1
    "Why doesn't the Decimal constructor use that trick too?" Evidently the designers of the `decimal` module thought strict accuracy was a better outcome. I don't know why they would think that, since the whole point of the module is to work with decimal numbers that can't be represented in binary. The number you see is exact by the way - a binary number can always be represented in decimal even though the reverse isn't true. – Mark Ransom Mar 14 '14 at 02:43

1 Answers1

4

The exact value of the float is 1.100000000000000088817841970012523233890533447265625. Python isn't somehow keeping track of the original string. When Python stringifies it with str, it truncates to 12 digits.

>>> x = 1.0/9
>>> print decimal.Decimal(x) # prints exact value
0.111111111111111104943205418749130330979824066162109375
>>> print x # truncated
0.111111111111

Even with repr, Python uses the shortest string that will round to the original float when parsed with float.

>>> print repr(x)
0.1111111111111111

If you want to parse JSON and get Decimal instances instead of float, you can pass a parse_float argument to the load(s) function:

>>> json.loads('{"foo": 1.234567890123456789}', parse_float=decimal.Decimal)
{u'foo': Decimal('1.234567890123456789')}

The above call causes decimal.Decimal to be called to parse numbers in the JSON string, bypassing the rounding that would occur with intermediate float or str calls. You'll get exactly the number specified in the JSON.

Note that the API distinguishes between the functions used to parse things that look like floats, things that look like ints, and things that look like Infinity, -Infinity, or NaN. This can be a bit inconvenient if you want all 3 categories to be handled the same way:

>>> json.loads('{"foo": 1.234567890123456789}',
...            parse_float=decimal.Decimal,
...            parse_int=decimal.Decimal,
...            parse_constant=decimal.Decimal)
{u'foo': Decimal('1.234567890123456789')}
user2357112
  • 260,549
  • 28
  • 431
  • 505
  • I wish I could give another +1 just for the `parse_float` reference, that looks really useful. – Mark Ransom Mar 14 '14 at 02:45
  • (OP here..) Yes looks like parse_float is how I should handle this. It's scientific data so I have to be super strict with handling numbers. My tool can accept a data structure or a JSON string, so I have to handle floats too... if I do get float values I think I'll convert them using `Decimal(repr(the_float))`, that seems to be as close to the original human intention as I can get. – Daniel Baird Mar 14 '14 at 02:55