Strange behavior of Numpy `where` clause

Question

I see a strange behavior with ufunc where clause for Numpy 1.15.3.

In [1]: import numpy as np

In [2]: x = np.array([[1,2],[3,4]])

In [3]: y = np.ones(x.shape) * 2

In [4]: print(x, "\n", y)
[[1 2]
 [3 4]]
 [[2. 2.]
 [2. 2.]]

In [5]: np.add(x, y, where=x==3)
Out[5]:
array([[2., 2.],     #<=========== where do these 2s come from???
       [5., 2.]])

In [6]: np.add(x, y, where=x==3, out=np.zeros(x.shape))
Out[6]:
array([[0., 0.],
       [5., 0.]])

In [7]: np.add(x, y, where=x==3, out=np.ones(x.shape))
Out[7]:
array([[1., 1.],
       [5., 1.]])

In [8]: np.add(x, y, where=x==3)
Out[8]:
array([[1., 1.], # <========= it seems these 1s are remembered from last computation.
       [5., 1.]])

ADD1

It seems I can only get the rational result with the out parameter.

Below is without the out parameter:

import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x = np.linspace(-2,2,60)
y = np.linspace(-2,2,60)

xx, yy = np.meshgrid(x,y)

r= np.ones((60,60), dtype=float) * 2
z = np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0) # <==== HERE!!

surf = ax.plot_surface(xx, yy, z, cmap="viridis")

This generates a ridiculous image:

If I add the out parameter as below, everything works fine.

z = np.zeros(xx.shape)
np.sqrt(r**2 - xx**2 - yy**2, where=(r**2 - xx**2 - yy**2)>=0, out=z)

*"where do these 2s come from???"* See @tel's answer. The 2's might be from a temporary array being created, freed, and then the same memory being reused for `out`. — Warren Weckesser, Nov 15 '18 at 05:19
This question looks like a duplicate of https://stackoverflow.com/questions/45543505/what-is-the-default-of-numpy-functions-with-where-false; also see https://stackoverflow.com/questions/49459985/numpy-reciprocal-returns-different-values-when-called-repeatedly — Warren Weckesser, Nov 15 '18 at 05:24

tel · Accepted Answer · 2018-11-15T05:45:08.550

You're ending up with junk data in your outputs due to the use of where. The fix, as you've said, is to initialize your own output and pass that to out.

From the docs about the out arg:

If ‘out’ is None (the default), a uninitialized return array is created. The output array is then filled with the results of the ufunc in the places that the broadcast ‘where’ is True. If ‘where’ is the scalar True (the default), then this corresponds to the entire output being filled. Note that outputs not explicitly filled are left with their uninitialized values.

So the values of out that you skip over (ie the indices in which where is False) are just left with whatever value was in them previously. That's why it looks like numpy is "remebering" values from previous computations, like the 1s at the end of your first example code block.

As @WarrenWeckesser points out in his comment, this also implies that the same memory block is being reused for the output when out is left blank, at least in some cases. Interestingly, you can alter the results you were getting by assigning each output to a variable:

x = np.array([[1,2],[3,4]])
y = np.ones(x.shape) * 2

arr0 = np.add(x, y, where=x==3)
arr1 = np.add(x, y, where=x==3, out=np.zeros(x.shape))
arr2 = np.add(x, y, where=x==3, out=np.ones(x.shape))
arr3 = np.add(x, y, where=x==3)
print(arr3)

Now you can clearly see the junk data in the output:

[[-2.68156159e+154 -2.68156159e+154]
 [ 5.00000000e+000  2.82470645e-309]]

Strange behavior of Numpy `where` clause

ADD1

1 Answers1