0

Can anyone explain why the following Ruby routine gives me a 2.0 as a result? I think rounding the floats is the root of the error.

puts(999_999_999_999_999_9.0 - 999_999_999_999_999_8.0);

= 2.0

The same error?

puts(999_999_999_999_999_3.0 - 999_999_999_999_999_2.0);

= 0.0

I'm running Ruby 1.9.3p448 (2013-06-27) [i386-mingw32].

the Tin Man
  • 158,662
  • 42
  • 215
  • 303
medi_kirk
  • 1
  • Yes this is floating point rounding. The numbers you have entered cannot be represented with complete accurately using Ruby's `Float` class. The imprecision is not spread evenly, and the representation of your second two numbers is actually identical - it is the same `Float` object. – Neil Slater Nov 10 '13 at 21:20

1 Answers1

0

These numbers are already too big and Ruby cannot count them in ones already - it can't represent so many ones. So, it count them in twos. Numbers twice bigger will be counted in fours, and so on.

That depends on the number of digits that can mantissa of a float number accept. It is always mentioned in the base types description.

Gangnus
  • 24,044
  • 16
  • 90
  • 149