I am trying to get a better understand of floating point arithmetic. I know machine epsilon (e) is define as the difference between 1 and the next largest number (i.e. the next largest number after 1 that can be accurately represented in floating point is 1+e).
However, what do I get in floating point when I multiply (1+e) * (1+e)? Theoretically it should be 1 + 2*e+ e^2, but (assuming e<1) e^2 < e so e^2 will not be completely accurate. What does this answer round to in floating point?
As noted in the comments eps^2 is exactly representable in any floating point system in question. But when added to anything >= 1 it will simply truncate out. There are insufficient bits of precision to include it and the "1.0" term pins the exponent. Therefore -
(1.0 + eps)^2 -> 1.0 + 2*eps
You will need to verify this on your hardware. I have learned the hard way that floating-point hardware does not always behave as one might expect and I have seen advanced numerical software execute initialization code not only to approximate a usable eps (not always the floating-point system defined one) but also carry out verification of eps behaviors that the algorithm is counting on to deliver correct results.
The pros never take anything for granted. Sometimes fp hardware is implemented incorrectly. Sometimes fp systems have odd boundary condition behavior. Trust but verify would be my recommendation.
