1

I have a numpy array which contains no data values. I mask those no data values so that they do not influence my calculations using:

    array = numpy.ma.masked_values(array, options['ndv'], copy=False)

I then use memmove to get the numpy array into a shared ctypes array using:

def ndarray_to_shmem(array):
    """ Converts a numpy.ndarray to a multiprocessing.Array object.

    The memory is copied, and the array is flattened.
    """
    arr = array.reshape((-1, ))
    data = RawArray(_numpy_to_ctypes[array.dtype.type], 
                                    arr.size)
    ctypes.memmove(data, array.data[:], len(array.data))
    return data

Which returns the following stack trace:

ctypes.memmove(data, array.data[:], len(array.data))
ctypes.ArgumentError: argument 2: <type 'exceptions.TypeError'>: wrong type

Is it possible to use memmove to move the masked array into a shared, ctypes array?

Jzl5325
  • 3,898
  • 8
  • 42
  • 62
  • What exactly are you expecting to happen when you move the masked array? Are you hoping to get only the values that are not masked? Should the no-value entries be converted to nan? – Luke Aug 07 '12 at 23:32
  • Converted to NaN. I will then convert back to the input's no data value after performing some calculation. – Jzl5325 Aug 08 '12 at 00:04

1 Answers1

2

First of all, you need to change this line:

ctypes.memmove(data, array.data[:], len(array.data))

to look like this:

ctypes.memmove(data, array.data[:].ctypes.data, len(array.data))

Second, ctypes.memmove has no understanding of masked arrays. Instead, just make a copy with the masked areas set to nan:

masked = array.copy()
masked[array == options['ndv']] = np.nan

...

ctypes.memmove(data, masked.ctypes.data, len(masked))
Luke
  • 11,374
  • 2
  • 48
  • 61
  • Luke, why the first item? Just stumbled onto the second via the numpy docs thanks to your initial comment. Thanks! – Jzl5325 Aug 08 '12 at 00:29
  • The requirement to use array.ctypes.data is because ctypes has no understanding of numpy arrays (given a numpy array, ctypes does not know how to locate the array data it contains). The attribute array.ctypes.data is just a pointer to the memory array contained within, which ctypes is able to understand. – Luke Aug 08 '12 at 00:36