Different numpy overflow behavior on mac vs linux

Question

I'm encountering different overflow behavior on Mac vs Linux with the same version of numpy. MWE:

import numpy as np
arr = np.arange(0, 2 * 4e9, 1e9, dtype=float)
print(arr.astype(np.uint32))
print(np.__version__)

Mac (Python 3.9.13):

array([         0, 1000000000, 2000000000, 3000000000, 4000000000,
        705032704, 1705032704, 2705032704], dtype=uint32)
'1.22.4'

Linux (Python 3.9.7):

array([         0, 1000000000, 2000000000, 3000000000, 4000000000,
                0,          0,          0], dtype=uint32)
'1.22.4'

I would prefer the "Mac" behavior of the expected rollover (rather than forcing overflowed values to 0), so I would like to know how to fix this for the Linux version.

FlyingTeller · Accepted Answer · 2023-08-18T08:07:29.970

I believe that this is due to the underlying C implementation of numpy, which probably triggers undefined behaviour, which is handled differently by the compiler used for the linux and mac distributions of numpy.

Looking at cast float to unsigned int in C with gcc which deals with a similar topic, we can also see link to the C standard which states on page 51

6.3.1.4 Real floating and integer
1 When a finite value of real floating type is converted to an integer type other than _Bool, the fractional part is discarded (i.e., the value is truncated toward zero). If the value of the integral part cannot be represented by the integer type, the behavior is undefined.61)

One possibility would be I guess that you try out different compilers in compiling your own numpy version and check the behaviour. Alternatively, since you want the roll-over behaviour, you could try to convert first to an unsigned integer type of larger width and then to the smaller width, since unsigned integer conversions always do a roll-over

6.3.1.3 Signed and unsigned integers
1 When a value with integer type is converted to another integer type other than _Bool, if the value can be represented by the new type, it is unchanged.
2 Otherwise, if the new type is unsigned, the value is converted by repeatedly adding or subtracting one more than the maximum value that can be represented in the new type until the value is in the range of the new type.60)

import numpy as np
arr = np.arange(0, 2 * 4e9, 1e9, dtype=float)
print(arr.astype(np.uint64).astype(np.uint32))
print(np.__version__)

Different numpy overflow behavior on mac vs linux

1 Answers1