57

is it possible to add extension method to python built-in types? I know that I can add extension method to defined type by simply adding new method by . as following:

class myClass:
    pass

myClass.myExtensionMethod = lambda self,x:x * 2
z = myClass()
print z.myExtensionMethod(10)

But is any way to adding extension method to python built'in types like list, dict, ...

list.myExtension = lambda self,x:x * 2
list.myExtension(10)
gerrit
  • 24,025
  • 17
  • 97
  • 170
Saeed Afshari
  • 939
  • 1
  • 9
  • 17

5 Answers5

81

It can be done in pure Python with this incredibly clever module:

https://pypi.python.org/pypi/forbiddenfruit

For example:

import functools
import ctypes
import __builtin__
import operator

class PyObject(ctypes.Structure):
    pass

Py_ssize_t = hasattr(ctypes.pythonapi, 'Py_InitModule4_64') and ctypes.c_int64 or ctypes.c_int

PyObject._fields_ = [
    ('ob_refcnt', Py_ssize_t),
    ('ob_type', ctypes.POINTER(PyObject)),
]

class SlotsPointer(PyObject):
    _fields_ = [('dict', ctypes.POINTER(PyObject))]

def proxy_builtin(klass):
    name = klass.__name__
    slots = getattr(klass, '__dict__', name)

    pointer = SlotsPointer.from_address(id(slots))
    namespace = {}

    ctypes.pythonapi.PyDict_SetItem(
        ctypes.py_object(namespace),
        ctypes.py_object(name),
        pointer.dict,
    )

    return namespace[name]

def die(message, cls=Exception):
    """
        Raise an exception, allows you to use logical shortcut operators to test for object existence succinctly.

        User.by_name('username') or die('Failed to find user')
    """
    raise cls(message)

def unguido(self, key):
    """
        Attempt to find methods which should really exist on the object instance.
    """
    return functools.partial((getattr(__builtin__, key, None) if hasattr(__builtin__, key) else getattr(operator, key, None)) or die(key, KeyError), self)

class mapper(object):
    def __init__(self, iterator, key):
        self.iterator = iterator
        self.key = key
        self.fn = lambda o: getattr(o, key)

    def __getattribute__(self, key):
        if key in ('iterator', 'fn', 'key'): return object.__getattribute__(self, key)
        return mapper(self, key)

    def __call__(self, *args, **kwargs):
        self.fn = lambda o: (getattr(o, self.key, None) or unguido(o, self.key))(*args, **kwargs)
        return self

    def __iter__(self):
        for value in self.iterator:
            yield self.fn(value)

class foreach(object):
    """
        Creates an output iterator which will apply any functions called on it to every element
        in the input iterator. A kind of chainable version of filter().

        E.g:

        foreach([1, 2, 3]).__add__(2).__str__().replace('3', 'a').upper()

        is equivalent to:

        (str(o + 2).replace('3', 'a').upper() for o in iterator)

        Obviously this is not 'Pythonic'.
    """
    def __init__(self, iterator):
        self.iterator = iterator

    def __getattribute__(self, key):
        if key in ('iterator',): return object.__getattribute__(self, key)
        return mapper(self.iterator, key)

    def __iter__(self):
        for value in self.iterator:
            yield value

proxy_builtin(list)['foreach'] = property(foreach)

import string

print string.join([1, 2, 3].foreach.add(2).str().add(' cookies').upper(), ', ')

>>> 3 COOKIES, 4 COOKIES, 5 COOKIES

There, doesn't that feel good?

toothygoose
  • 1,292
  • 1
  • 10
  • 9
  • 7
    This reminds one of [that scene](http://imgur.com/lVEdIXO) in The Empire Strikes Back.... – n611x007 Apr 29 '14 at 17:00
  • 35
    What on earth is all of the code in this answer? Just do `pip install forbiddenfruit`, then `from forbiddenfruit install curse` and set about destroying the universe. – ArtOfWarfare Jul 03 '15 at 22:20
  • @ArtOfWarfare he added cookies so I think it was worth it – MaLiN2223 Feb 01 '18 at 10:25
  • 4
    `forbiddenfruit` only knows how to penetrate a class's dict proxy to mess with the underlying dict. It doesn't know how to fix any of the other things it breaks, like the type attribute cache or the C-API slots for operator overloads. It's very easy to segfault Python or get it into an inconsistent state where operations that should invoke the same method do different things, if you use forbiddenfruit. – user2357112 Oct 21 '18 at 17:26
20

No. Types defined in C cannot be monkeypatched.

Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358
  • 1
    It's sad but true. I usually end up inheriting from the builtin and monkey patch the child class if it becomes necessary. – Ishpeck Jul 18 '11 at 20:34
  • 4
    @Ishpeck: That sounds like ordinary subclassing, not monkey patching at all. – S.Lott Jul 18 '11 at 20:39
  • 2
    It's usually monkeypatching, just done to the instance of the subclass. And it's crufty and bad. Don't ever do it, kids. – Ishpeck Jul 18 '11 at 20:48
  • 4
    @Ishpeck: Why not simply apply the change when subclassing? – S.Lott Jul 18 '11 at 20:52
  • Like I said. Crufty and bad. – Ishpeck Jul 18 '11 at 21:01
  • 1
    @Ishpeck: Like I said. Why? "Crufty and bad" is not really much of a reason. – S.Lott Jul 18 '11 at 21:08
  • It's not exactly monkeypatching. It's more like syntactic gravy. My reasoning: Extension methods in c# are basically global static functions that, using special notation, are associated with a given type. Then, you can use the dot notation to call that global static function off of that object and the compiler knows to just pass that object as the first argument to that function. – eremzeit Aug 13 '11 at 23:20
  • @eremzeit Which is a fantastic way to do "monkey patching" –  Apr 15 '13 at 15:56
  • See the if you see monkey patching for built in types https://github.com/emre/unicode_tr/pull/5/files – guneysus Nov 20 '15 at 15:51
  • @S.Lott Subclassing doesn't work for objects that you create through a factory instead of a constructor. – Damian Yerrick Mar 25 '17 at 22:21
12

Nope, you gotta subclass!

>>> import string
>>> class MyString(str):
...     def disemvowel(self):
...         return MyString(string.translate(self, None, "aeiou"))
... 
>>> s = MyString("this is only a test")
>>> s.disemvowel()
'ths s nly  tst'

Or more specific to your example

>>> class MyList(list):
...     pass
... 
>>> MyList.myExtension = lambda self,x:x * 2
>>> l = MyList()
>>> l.myExtension(10)
20
ironchefpython
  • 3,409
  • 1
  • 19
  • 32
3

No, because I'm pretty sure all the built-in types are written in optimized C and thus can't be modified with Python. When I try it, I just get:

TypeError: can't set attributes of built-in/extension type 'list'
fletom
  • 1,998
  • 13
  • 17
2

The best you can do appears to be deriving a class from the built-in type. For example:

class mylist(list):
    def myfunc(self, x):
        self.append(x)

test = mylist([1,2,3,4])
test.myfunc(99)

(You could even name it "list" so as to get the same constructor, if you wanted.) However, you cannot directly modify a built-in type like the example in your question.

user812786
  • 4,302
  • 5
  • 38
  • 50