2

I would like to add a property (.csys) to a subclass of numpy.ndarray:

import numpy as np

class Point(np.ndarray):

    def __new__(cls, arr, csys=None):
        obj = np.asarray(arr, dtype=np.float64).view(cls)
        obj._csys = csys
        return obj

    def __array_finalize__(self, obj):
        if obj is None: return
        self._csys = getattr(obj, '_csys', None)

    @property
    def csys(self):
        print('Getting .csys')
        return self._csys

    @csys.setter
    def csys(self, csys):
        print('Setting .csys')
        self._csys = csys

However, when I run this test code:

pt = Point([1, 2, 3])
pt.csys = 'cmm'
print("pt.csys:", pt.csys)

# Pickle, un-pickle, and check again
import pickle

pklstr = pickle.dumps(pt)
ppt = pickle.loads(pklstr)

print("ppt.csys:", ppt.csys)

it appears that the attribute cannot be pickled:

Setting .csys
Getting .csys
pt.csys: cmm
Getting .csys
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
C:\Rut\Vanes\bin\pointtest.py in <module>()
     39     ppt = pickle.loads(pklstr)
     40 
---> 41     print("ppt.csys:", ppt.csys)

C:\Rut\Vanes\bin\point.py in csys(self)
     15     def csys(self):
     16         print('Getting .csys')
---> 17         return self._csys
     18 
     19     @csys.setter

AttributeError: 'Point' object has no attribute '_csys'

I tried doing the same thing without using decorators (e.g. defining get_csys() and set_csys(), plus csys = property(__get_csys, __set_csys), but had the same result with that.

I'm using numpy 1.13.3 under Python 3.6.3

subnivean
  • 1,132
  • 11
  • 19
  • Cannot reproduce... this works for me on python 3.6.0 and numpy 1.14.0. Are you sure you're using python3? – cs95 May 17 '18 at 04:48
  • The opening banner when I run Jupyter QtConsole 4.3.1 tells me so, yes. Is it a numpy 1.13 problem? – subnivean May 17 '18 at 04:58
  • I don't know. You should try to upgrade and see if it persists. – cs95 May 17 '18 at 04:59
  • [This](https://stackoverflow.com/questions/26598109/preserve-custom-attributes-when-pickling-subclass-of-numpy-array) seems to be related. In conclusion, subclassing `__reduce__` and `__setstate__` is needed to tell the ndarray subclass how to pickle itself in diff to a normal ndarray. – Jeronimo May 17 '18 at 05:25
  • @Jeronimo Yes, thanks, just found that myself off to the right and it works perfectly. I'll write that up as my own answer soon. – subnivean May 17 '18 at 05:34

1 Answers1

0

This question has already been asked and answered here. In a nutshell, numpy uses __reduce__ and __setstage__ to pickle itself. The overrides, adapted to the case above, look like this:

def __reduce__(self):
    # Get the parent's __reduce__ tuple
    pickled_state = super().__reduce__()
    # Create our own tuple to pass to __setstate__
    new_state = pickled_state[2] + (self._csys,)
    # Return a tuple that replaces the parent's __setstate__ tuple with our own
    return (pickled_state[0], pickled_state[1], new_state)

def __setstate__(self, state):
    self._csys = state[-1]  # Set the _csys attribute
    # Call the parent's __setstate__ with the other tuple elements.
    super().__setstate__(state[0:-1])

Also note that the getter and setter methods (under the @property and @csys.getter decorators, respectively) are not strictly required in this simple case. If they are dispensed with, access .csys directly, rather than through the 'private' ._csys attribute.

subnivean
  • 1,132
  • 11
  • 19