1

When I run these simple sums, I get intriguing errors:

import numpy as np

data_type = np.float16
x = np.ones(shape=(3, 1000,10), dtype=data_type)
y = np.ones(shape=(3, 10000, 10), dtype=data_type)
z = np.ones(shape=(3, 11053, 10), dtype=data_type)

print("\nx: Both results are correct")
print(f"x[0,:,0].sum(0)={x[0,:,0].sum(0)}")
print(f"x[0].sum(0)[0]={x[0].sum(0)[0]}")

print("\ny: One result is correct, one is wrong")
print(f"y[0,:,0].sum(0)={y[0, :, 0].sum(0)}")
print(f"y[0].sum(0)[0]={y[0].sum(0)[0]}")

print("\nz: Both results are wrong")
print(f"z[0,:,0].sum(0)={z[0, :, 0].sum(0)}")
print(f"z[0].sum(0)[0]={z[0].sum(0)[0]}")

Output:

> x: Both results are correct
> x[0,:,0].sum(0)=1000.0
> x[0].sum(0)[0]=1000.0

> y: One result is correct, one is wrong
> y[0,:,0].sum(0)=10000.0
> y[0].sum(0)[0]=2048.0

> z: Both results are wrong
> z[0,:,0].sum(0)=11056.0
> z[0].sum(0)[0]=2048.0

The errors do not occur when using np.float32 or np.float64. However, according to np.finfo(np.float16).max and np.finfo(np.float16).min, the maximal and minimal values of np.float16 are +/- 65,500 so it doesn't seem to be due to an overflow. Also, in my second example (with y) one sum result is correct and the other is not, which seems to point out that the indexing order makes a difference.

Any idea on what is going on?

Julep
  • 760
  • 1
  • 6
  • 18
  • 1
    Does this answer your question? [Is float16 datatype for numpy disfunctional?](https://stackoverflow.com/questions/40645537/is-float16-datatype-for-numpy-disfunctional) – Alistair Mar 20 '20 at 17:17
  • Partly, and thanks for the pointer! However, it does not answer why the indexing/reduction order changes the answer. – Julep Mar 20 '20 at 17:45
  • Because the way it got summed @Julep, the second one summed rows up to 2048 then starting adding ones to this sum, in the period from 2048 to 4096 only even numbers are exactly represented and odd numbers are rounded to nearest even, so adding 2048 +1 = 2048 , doing this repeatedly for a million time won't change anything – kareem_emad Mar 20 '20 at 18:55

0 Answers0