python - modify part of a masked array inside a function

Question

I need to modify part of a masked array inside a function eg:

import numpy.ma as ma

arr_2d = ma.masked_all((5,5))
arr_3d = ma.masked_all((5,5,5))
arr_3d[0,1] = 5

def func1(arr, val):
    arr[:] = val

looks simple enough but then...

>>> func1(arr_3d[0], 1)
>>> arr_3d[0]
masked_array(data =
 [[-- -- -- -- --]
 [1.0 1.0 1.0 1.0 1.0]
 [-- -- -- -- --]
 [-- -- -- -- --]
 [-- -- -- -- --]],
             mask =
 [[ True  True  True  True  True]
 [False False False False False]
 [ True  True  True  True  True]
 [ True  True  True  True  True]
 [ True  True  True  True  True]],
       fill_value = 1e+20)

it seems to be something to do with the sharedmask always being set on a slice of the array so that the mask is passed to the function as a copy

I'm hoping there might be some way to fix or get around it other than explicitly passing the mask, returning a copy of the data or passing the larger array with an index.

Wouldn't you be better off returning `arr` from `func1` and then doing `arr_3d[0] = func1(arr_3d[0], 1)`? — Jack Yates, Aug 25 '16 at 14:27

hpaulj · Accepted Answer · 2016-08-25T17:12:06.893

The warning in a recent numpy is:

In [738]: func1(A[1],1)
/usr/local/bin/ipython3:2: MaskedArrayFutureWarning: setting an item on 
a masked array which has a shared mask will not copy the mask and also 
change the original mask array in the future.
Check the NumPy 1.11 release notes for more information.

http://docs.scipy.org/doc/numpy/release.html#assigning-to-slices-views-of-maskedarray

Currently a slice of a masked array contains a view of the original data and a copy-on-write view of the mask. Consequently, any changes to the slice’s mask will result in a copy of the original mask being made and that new mask being changed rather than the original.

After this action, row 1 of A is still masked, but the A[,:].data` has been changed.

In [757]: B=np.ma.masked_all((5))
...
In [759]: B[0]=5     # direct __setitem__ change to B
In [760]: B
Out[760]: 
masked_array(data = [5.0 -- -- -- --],
             mask = [False  True  True  True  True],
       fill_value = 1e+20)
In [761]: func1(B[3:],1)
/usr/local/bin/ipython3:2: MaskedArrayFutureWarning: ....

In [762]: B      # no change to mask
Out[762]: 
masked_array(data = [5.0 -- -- -- --],
             mask = [False  True  True  True  True],
       fill_value = 1e+20)
In [763]: B.data      # but data is changed
Out[763]: array([ 5.,  0.,  0.,  1.,  1.])

A[1,:]=1 is a direct use of the masked __setitem__, and it can take full responsibility for setting both the data and mask. In your function A is a view of the original, obtained with a A.__getitem__ call. Apparently developers have worried about whether changes to this view`s mask should affect the mask of the original or not.

We may have to look developers discussions; the warning indicates that something was changed recently.

============

The issue isn't about use in a function, it's about a view

In [764]: B1=B[3:]
In [765]: B1[:]=2
/usr/local/bin/ipython3:1: MaskedArrayFutureWarning:...
In [766]: B
Out[766]: 
masked_array(data = [5.0 -- -- -- --],
             mask = [False  True  True  True  True],
       fill_value = 1e+20)
In [767]: B.data
Out[767]: array([ 5.,  0.,  0.,  2.,  2.])

The warning describes what is happening now, and possibly for some time. It's saying that this practice will change.

Following the change notes suggestion:

In [785]: B1=B[3:]
In [787]: B1._sharedmask
Out[787]: True
In [790]: B1._sharedmask=False
In [791]: B1[:]=4
In [792]: B1
Out[792]: 
masked_array(data = [4.0 4.0],
             mask = [False False],
       fill_value = 1e+20)
In [793]: B     # mask has been changed along with data
Out[793]: 
masked_array(data = [5.0 -- -- 4.0 4.0],
             mask = [False  True  True False False],
       fill_value = 1e+20)

So changing

 def func1(arr,val):
     arr._sharedmask=False
     arr[:]=val

will stop the warning, and modify the mask of the original array.

python - modify part of a masked array inside a function

1 Answers1