Im trying to figure why the number 1 + 2.0^-24 in single arithmetic precision is stored as 1.0 instead of 1 + 2^-23 since we can check that 1 + 2.0^-24 - 1 + 2^-23 = 2^-24 and 1.0 + 2.0^-24 - 1.0 = 2^-24 and 1.0 and 1.0 + 2^-23 are two closet numbers to 1.0+2^-24. Why is the computer choosing 1.0 ?
Asked
Active
Viewed 21 times
0
-
Thats very dependant. What data type does the variable you store it in have? – Magisch Jan 20 '16 at 09:59
-
It is based on your rounding principle – padawan Jan 20 '16 at 10:00
-
The programming language, data type, compiler, and processor type (64 vs 32 bit) may all have an influence on how data is actually stored, as far as I know. – TobiMcNamobi Jan 20 '16 at 10:06