0

Something very weird is happening with decimals and floating numbers and I can't understand why or where (ruby/rails/postgresql).

Given a purchases table with a decimal column - total:

p1 = Purchase.where(total: 5.99).first_or_create
p2 = Purchase.where(total: 5.99).first_or_create

[p1.id, p2.id] # => [1, 2]

p3 = Purchase.where(total: 5.99.to_d).first_or_create
p4 = Purchase.where(total: 5.99.to_d).first_or_create

[p3.id, p4.id] # => [1, 1]

Both Ruby and postgresql have no problem representing 5.99 exactly, no matter if decimals or floats:

5.99.to_s         # => "5.99"
5.99.to_d.to_s    # => "5.99"
5.99 == 5.99.to_d # => true

SELECT CAST(5.99 AS DECIMAL) AS decimal, CAST(5.99 AS FLOAT) AS float;
  #  decimal | float 
  # ---------+-------
  #     5.99 |  5.99
  # (1 row)

SELECT CAST(5.99 AS DECIMAL) = CAST(5.99 AS FLOAT) AS equal;
  #  equal 
  # -------
  #  t
  # (1 row)

To top it all off, this doesn't happen with some other values:

p5 = Purchase.where(total: 5.75).first_or_create
p6 = Purchase.where(total: 5.75).first_or_create
p7 = Purchase.where(total: 5.75.to_d).first_or_create

[p5.id, p6.id, p7.id] # => [3, 3, 3]
ndnenkov
  • 35,425
  • 9
  • 72
  • 104
  • Basically, you never want your `total` to ever be a floating point value anywhere, you'd want to be saying things like `'5.99'.to_d` instead. – mu is too short Nov 25 '16 at 17:24
  • @muistooshort, practical solutions are great, but answering a *why* question with *"don't do it"* is counter productive. You don't even know if they are same issue. Firstly, the other question starts with two different numbers. Secondly, two digit precision should be something that floats can handle without problem. Thirdly, you can see the problem is in Ruby there, while here we are not able to detect the issue (as shown in the examples). – ndnenkov Nov 25 '16 at 18:18
  • @muistooshort, Please reopen this. The reason I created this question is because I want to get a greater understanding of what is happening. If I wanted a way for things to just work - I have already provided a solution in the question. – ndnenkov Nov 25 '16 at 18:22
  • Sure but it is almost certainly exactly the same underlying problem: `BigDecimal` does weird things with some floats. The _why_ part is covered by all kinds of other floating point questions. I think you're mistaken about the "two digit precision" stuff, floats are internally represented in binary and some decimal representations cannot be represented in binary with full precision regardless of the _decimal_ precision, your `5.75` example can be because of the properties of `3/4`, `5.99` isn't so simple. – mu is too short Nov 25 '16 at 19:24
  • For reference: this was the duplicate: [Why are my BigDecimal objects initialized with unexpected rounding errors?](http://stackoverflow.com/q/28295583/479863). – mu is too short Nov 25 '16 at 19:27
  • @muistooshort, *x does weird things with y* is such a blanket term. The issue in the other question was demonstrable to be reproducible inside Ruby, while here that is unclear. Thank you for being reasonable and reopening the question. – ndnenkov Nov 25 '16 at 19:37
  • 2
    No binary floating point system can represent 5.99 exactly. The closest IEEE 754 64-bit binary floating point value is 5.9900000000000002131628207280300557613372802734375 – Patricia Shanahan Nov 26 '16 at 01:59
  • @PatriciaShanahan, fair point, that is probably part of the puzzle. – ndnenkov Nov 29 '16 at 08:20

1 Answers1

0

This turned out to be a regression in rails. It's reproducible with 5.0.0.1??? and is gone by 5.1.0.0???.


I bisected it and found this commit to be the one that fixes the issue. This is the related issue.

The fix seems to be to stop using the pg gem's float encoder.

ndnenkov
  • 35,425
  • 9
  • 72
  • 104