-2

Can someone explain this:

>>> numpy.array([22.0], dtype=numpy.float64) / (2 ** 11)
array([0.01074219])
>>> 22.0 / (2 ** 11)
0.0107421875
>>>

Numpy seems to generates an erroneous result, probably some kind of precision error.

How can I fix this?

Mark Dickinson
  • 29,088
  • 9
  • 83
  • 120
Julien REINAULD
  • 599
  • 2
  • 5
  • 18
  • The last non-zero digit to the right of the decimal point for non whole number floats is always 5. I find this factoid (not sure where it is written; i sort of postulated it myself) useful. – Andrew Aug 08 '21 at 18:45

1 Answers1

1
>>> numpy.set_printoptions(precision=15)
>>> numpy.array([22.0], dtype=numpy.float64) / (2 ** 11)
array([0.0107421875])

fixes this

Julien REINAULD
  • 599
  • 2
  • 5
  • 18