2

Hi I just ran into an issue where ruby's to_f function is giving me inconsistent results.

ruby-1.9.2-head :026 > 8.45.to_f * 100
 => 844.9999999999999 

ruby-1.9.2-head :027 > 4.45.to_f * 100
 => 445.0 


ruby-1.9.2-head :028 > 4.35.to_f * 100
 => 434.99999999999994

My workaround is to simply round the result this way

ruby-1.9.2-head :029 > (4.35.to_f * 100).round
 => 435

After more playing around I realised that the issue might be with the multiplication operator * 100

antyrat
  • 27,479
  • 9
  • 75
  • 76
NoTiTo
  • 21
  • 2

4 Answers4

6

Welcome to floating point drift. This is a well understood problem, and you should have a read so you at least understand it yourself. For instance, have a peek over at the following article:

jer
  • 20,094
  • 5
  • 45
  • 69
  • Thanks for the link. Will give it read to try and get e better understanding. I see the section on "Rounding Error" mentions that you should round the result of a floating point calculation. =>"...Therefore the result of a floating-point calculation must often be rounded in order to fit back into its finite representation..." – NoTiTo Oct 24 '11 at 18:40
1

The problems with Float are already mentioned. See check the other answers.

Some more remarks:

You wrote 4.35.to_f. The to_f is not necessary in this case. 4.35 is already a Float:

p 4.35.class #-> Float

Where did you recognize the problem. When you print the number the value is already rounded. With String#% you can determine the details level of the output:

p 8.45.to_f * 100 #->845.0
p "%.12f" % (8.45.to_f * 100) # -> "845.000000000000"
p "%.13f" % (8.45.to_f * 100) # -> "844.9999999999999"
p "%.14f" % (8.45.to_f * 100) # -> "844.99999999999989"
p "%.16f" % (8.45.to_f * 100) # -> "844.9999999999998900"
knut
  • 27,320
  • 6
  • 84
  • 112
  • Thanks for you response. I use to_f because the number is actually user submitted and comes through as a string "4.35". I guess I should have indicated that part.. – NoTiTo Oct 24 '11 at 18:32
0

Alas, this is part of the curse of floating point math, and not just a problem in Ruby:

http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding

If you need exact arithmetic with decimals, use BigDecimal:

require 'bigdecimal'

(BigDecimal('4.35') * 100).to_f
  #=> 435.0
Alex Peattie
  • 26,633
  • 5
  • 50
  • 52
0

The fundamental problem is that the fraction 45/100 does not have an exact representation as a sequence of 1/2n terms. In fact, most fractions written with a small number of base-10 digits do not have an exact FP representation.

As a result, the actual number you get is a very close but not exact approximation to your base-10 number. The output results will depend on where you round to, but will be correct if you do anything the least bit reasonable when rounding.

If you don't round, the exact number you get will depend on where the fraction gets chopped off and on how many digits you attempt to convert. Where the fraction is chopped will depend on how many bits are needed to represent the mantissa. That's why you get different results for x.45 depending on x.

This question comes up all the time on stack overflow. I guess we need a floating-point-faq.

Ironically, every (in range) integer value does have an exact floating point format representation.

DigitalRoss
  • 143,651
  • 25
  • 248
  • 329