How to set default behaviors of magic methods in python?

Question

Suppose I want to have a wrapper class Image for numpy array. My goal is to let it behave just like a 2D array but with some additional functionality (which is not important here). I am doing so because inheriting numpy array is way more troublesome.

import numpy as np


class Image(object):
    def __init__(self, data: np.ndarray):
        self._data = np.array(data)

    def __getitem__(self, item):
        return self._data.__getitem__(item)

    def __setitem__(self, key, value):
        self._data.__setitem__(key, value)

    def __getattr__(self, item):
        # delegates array's attributes and methods, except dunders.
        try:
            return getattr(self._data, item)
        except AttributeError:
            raise AttributeError()

    # binary operations
    def __add__(self, other):
        return Image(self._data.__add__(other))

    def __sub__(self, other):
        return Image(self._data.__sub__(other))

    # many more follow ... How to avoid this redundancy?

As you can see, I want to have all the magic methods for numeric operations, just like a normal numpy array, but with the return values as Image type. So the implementations of these magic methods, i.e. the __add__, __sub__, __truediv__ and so on, are almost the same and it's kind of silly. My question is if there is a way to avoid this redundancy?

Beyond what specifically I am doing here, is there a way to code up the magic methods in one place via some meta-programming technique, or it's just impossible? I searched some about python metaclass, but it's still not clear to me.

Notice __getattr__ won't handle delegates for magic methods. See this.

Edit

Just to clarify, I understand inheritance is a general solution for a problem like this, though my experience is very limited. But I feel inheriting numpy array really isn't a good idea. Because numpy array needs to handle view-casting and ufuncs (see this). And when you use your subclass in other py-libs, you also need to think how your array subclass gets along with other array subclasses. See my stupid gh-issue. That's why I am looking for alternatives.

How is inheritance more troublesome? It sounds like the better way here. — AKX, Jul 25 '19 at 13:01
Because numpy constructs new array-like objects in ways more than normal python types (an array can be view-cast). See [doc](https://docs.scipy.org/doc/numpy/user/basics.subclassing.html) for more. — Chen, Jul 25 '19 at 13:10
You might want to take a look at the `__array_function__` protocol (you'd need a recent numpy though) — ev-br, Jul 25 '19 at 13:24
Because of the awkwardness of inheriting from `ndarray`, or writing a class such as yours, there's more of a tendency to use a few functions for specialized array actions, and to reserve the class approach to bigger projects where you can take the time to do a comprehensive job. `np.matrix` and `np.ma` are non-trivial examples of subclassing `ndarray`. Python has mix-ins for its collections, but I'm not aware of anything for `numpy`. — hpaulj, Jul 25 '19 at 16:03

AltF4 · Accepted Answer · 2019-07-26T18:34:49.323

The magic methods are always looked up in the class and bypass getattribute entirely so you must define them in the class. https://docs.python.org/3/reference/datamodel.html#special-lookup

However, you can save yourself some typing:

import operator
def make_bin_op(oper):
    def op(self, other):
        if isinstance(other, Image): 
            return Image(oper(self._data, other._data))
        else:
            return Image(oper(self._data, other))
    return op

class Image:
    ...
    __add__ = make_bin_op(operator.add)
    __sub__ = make_bin_op(operator.sub)

If you want you could make a dict of operator dunder names and the corresponding operators and add them with a decorator. e.g.

OPER_DICT = {'__add__' : operator.add, '__sub__' : operator.sub, ...}
def add_operators(cls):
    for k,v in OPER_DICT.items():
        setattr(cls, k, make_bin_op(v))

@add_operators
class Image:
    ...

You could use a metaclass to do the same thing. However, you probably don't want use a metaclass unless you really understand what's going on.

score 0 · Answer 2 · answered Jul 25 '19 at 13:08

What you're after is the concept called inheritance, a key part of object-oriented programming (see Wikipedia here.

When you define your class with class Image(object):, what that means is that Image is a subclass of object, which is a built-in type that does very little. Your functionality is added on to that more-or-less blank concept. But if instead you defined your class with class Image(np.array):, then Image would be a subclass of array, which means it would inherit all the default functionality of the array class. Essentially, any class method you want to leave as is you simply shouldn't redefine. If you don't write a __getitem__ function, it uses the one defined in array.

If you need to add additional functionality in any of those functions, you can still redefine them (called overriding) and then use super().__getitem__ (or whatever) to access the function defined in the inherited class. This often happens with __init__ for example.

For a more thorough explanation, take a look at the chapter on inheritance in Think Python.

`np.array` is a function, not a class. The class is `np.ndarray`. But it defines it's own `__new__`, not just `__init__`. So inheritance from `ndarray` is more complicated than in the usual Python case. https://docs.scipy.org/doc/numpy/user/basics.subclassing.html — hpaulj, Jul 25 '19 at 18:04

How to set default behaviors of magic methods in python?

Edit

2 Answers2