0

Is it possible for ray (python) to handle operations/stats over masked np.arrays? [np.nansum(...) in this case].

For non-masked arrays works perfectly; but it crashes for masked arrays (see message error below).

import numpy as np
import numpy.ma as ma
import ray
ray.init()

#-to nansum over axis 0 (layers)
@ray.remote
def sumo(x):
    return np.nansum(x,axis=0,keepdims=True)

#-to allocate/reference the 3D-np.array into ray 'cores' 
@ray.remote
def matrox(wat):
    return ma.copy(wat)

#-a small reproducible example
data = np.int16((np.random.normal(size=(4,5,6))*100))
#-masking the data
data = ma.masked_less_equal(data,0)
#-this fails:
data_id = matrox.remote(data)
##-this doesn't fail as it gets rid off of the mask (but I DO need the mask)
#data_id = matrox.remote(data.data)

data_id = ray.put(data)
answer = ray.get(sumo.remote(data_id))

The following is the error I get when running the line:

data_id = matrox.remote(data)

enter image description here

The size of the array is 4x5x6=120, but it 'complains' when fitting 120-values into 240=4x5x12 ValueError: cannot reshape array of size 120 into shape (4,5,12)

Manuel F
  • 145
  • 2
  • 7
  • 1
    copy and paste error. Don't post a picture of the error – Jake P Jul 31 '19 at 14:43
  • 2
    Only `np.ma` functions and methods reliably work with masked arrays. Other numpy code (and potentially third party code) works if it delegates to `ma` methods. Usually masked arrays are `filled` or `compressed` if they need to be passed to other code. – hpaulj Jul 31 '19 at 15:13
  • thanks @hpaulij. so that means it's not yet solvable this issue?. – Manuel F Jul 31 '19 at 18:00

0 Answers0