Monkeypatch with instance method

Question

I'm trying to monkeypatch how pandas Panel's slicing (__getitem__). This is straightforward to do with a basic function, foo.

from pandas import Panel
Panel.__getitem__ = ORIGINAL_getitem


def newgetitem(panel, *args, **kwargs):
    """ Append a string to return of panel.__getitem__"""
    out = super(Panel, panel).__getitem__(*args, **kwargs)
    return out+'custom stuff added'

Panel.__getitem__ = newgetitem

WhereORIGINAL_getitem is storing the original Panel method. I'm trying to extend to the case where foo() is not a function, but an instance method of an object, Foo. For example:

class Foo:

    name = 'some name'

    def newgetitem(self, panel, *args, **kwargs):
        """ Append a string to return of panel.__getitem__,
        but take attributes from self, like self.name
        """
        out = super(Panel, panel).__getitem__(*args, **kwargs)
        return out+'custom stuff added including name' + self.name

Foo.foo() must access the attribute self.name. Therefore, the monkeypatched function would need a reference to the Foo instance somehow, in addition to the Panel. How can I monkepatch panel with Foo.foo() and make self.name accessible?

The switching between the monkey patched function happens in another method, Foo.set_backend()

class Foo:

    name = 'some name'

    def foo(self):
        return 'bar, called by %s' % self.name

    def set_backend(self, backend):
        """ Swap between new or original slicing."""
        if backend != 'pandas':
            Panel.__getitem__ = newgetitem            
        else:
            Panel.__getitem__ = ORIGINAL_getitem

What I really need is for newgetitem to maintain a reference to self.

Solution Attempts

So far I've tried taking making newgetitem() a pure function, and using partial functions to pass a reference to self in. This doesn't work. Something like:

import functools

def newgetitem(foo_instance, panel, *args, **kwargs):
    ....

class Foo:

    ...
    def set_backend(self, backend):
        """ Swap between new or original slicing."""
        if backend != 'pandas':
            partialfcn = functools.partial(newgetitem, self)
            Panel.__getitem__ = partialfcn            
        else:
            Panel.__getitem__ = ORIGINAL_getitem

But this doesn't work. A reference to self is passed, but no access from the calling Panel possible. That is:

 panel['50']

Passes a reference to Foo, not to Panel.

Yes, I know this is bad practice, but it's just a workaround for the time-being.

score 1 · Answer 1 · answered Feb 27 '15 at 13:49

You can use patch from mock framework to handle your case. Even it is designed for testing, its primary work is monkey patching in defined contex.

Your set_backend() method could be:

def set_backend(self, backend):
    if backend != 'pandas' and self._patched_get_item is None:
        self._patched_get_item = patch("pandas.Panel.__getitem__", autospec=True, side_effect=self._getitem)
        self._patched_get_item.start()
    elif backend == 'pandas' and self._patched_get_item is not None:
        self._patched_get_item.stop()
        self._patched_get_item = None

That will work either when self._getitem is a method or a reference to a function.

score 1 · Answer 2 · answered Feb 27 '15 at 14:57

One way to do this is to create a closure (a function with reference to names other than locals or globals). A simple closure:

def g(x):
    def f():
        """f has no global or local reference to x, but can refer to the locals of the 
        context it was created in (also known as nonlocals)."""
        return x
    return f

func = g(1)
assert func() == 1

I don't have pandas on my system, but it works much the same with a dict.

class MyDict(dict):
    pass

d = MyDict(a=1, b=2)
assert d['a'] == 1

class Foo:

    name = 'name'

    def create_getitem(fooself, cls):
        def getitem(self, *args, **kwargs):
            out = super(cls, self).__getitem__(*args, **kwargs)
            return out, 'custom', fooself.name 
            # Above references fooself, a name that is not defined locally in the 
            # function, but as part of the scope the function was created in.
        return getitem

MyDict.__getitem__ = Foo().create_getitem(MyDict)
assert d['a'] == (1, 'custom', Foo.name)

print(d['a'])

score 0 · Answer 3 · answered Feb 27 '15 at 12:15

The basics of monkey patching are straightforward but it can quickly become tricky and subtle, especially if you're aiming at finding a solution that would work for both Python 2 and Python 3.

Furthermore, quickly hacked solutions are usually not very readable/maintenable, unless you manage to wrap the monkey patching logic nicely.

That's why I invite you to have a look at a library that I wrote especially for this purpose. It is named Gorilla and you can find it on GitHub.

In short, it provides a cool set of features, it has a wide range of unit tests, and it comes with a fancy doc that should cover everything you need to get started. Make sure to also check the FAQ!

Monkeypatch with instance method

Solution Attempts

3 Answers3