2

Using Microsoft Excel 2010.

Why do these two formulae produce different results?

= (0.5 - 0.4 - 0.1)

produces -2.775E-17 while

= 0.5 - 0.4 - 0.1

produces exactly 0.

The only difference is in the brackets. Does Excel try to do clever things with floating-point numbers that it thinks should be a certain value instead of another value in order to hide the usual problems of binary floating-point and make it look as though it's using decimal arithmetic? Is this documented?

DodgyCodeException
  • 5,963
  • 3
  • 21
  • 42
  • 1
    Consider reading this: [link](https://www.microsoft.com/en-us/microsoft-365/blog/2008/04/10/understanding-floating-point-precision-aka-why-does-excel-give-me-seemingly-wrong-answers/). – Pspl Nov 16 '18 at 11:54
  • 2
    Related: https://stackoverflow.com/q/36452194/270986, https://stackoverflow.com/q/40293983/270986. The answers to your last two questions appear to be (1) Yes, and (2) Not really, no. – Mark Dickinson Nov 16 '18 at 16:48
  • @MarkDickinson thanks. I think those two linked posts together fully answer my question in a roundabout way. – DodgyCodeException Nov 19 '18 at 09:37

0 Answers0