4

I'm running Python 2.7.10.

I need to intercept changes in a list. By "change" I mean anything that modifies the list in the shallow sense (the list is not changed if it consists of the same objects in the same order, regardless of the state of those objects; otherwise, it is). I don't need to find out how the list has changed, only that it has. So I just make sure I can detect that, and let the base method do its work. This is my test program:

class List(list):
    def __init__(self, data):
        list.__init__(self, data)
        print '__init__(', data, '):', self

    def __getitem__(self, key):
        print 'calling __getitem__(', self, ',', key, ')',
        r = list.__getitem__(self, key)
        print '-->', r
        return r

    def __setitem__(self, key, data):
        print 'before __setitem__:', self
        list.__setitem__(self, key, data)
        print 'after  __setitem__(', key, ',', data, '):', self

    def __delitem__(self, key):
        print 'before __delitem__:', self
        list.__delitem__(self, key)
        print 'after  __delitem__(', key, '):', self

l = List([0,1,2,3,4,5,6,7]) #1
x = l[5]                    #2
l[3] = 33                   #3
x = l[3:7]                  #4
del l[3]                    #5
l[0:4]=[55,66,77,88]        #6
l.append(8)                 #7

Cases #1, #2, #3, and #5 work as I expected; #4, #6, and #7 don't. The program prints:

__init__( [0, 1, 2, 3, 4, 5, 6, 7] ): [0, 1, 2, 3, 4, 5, 6, 7]
calling __getitem__( [0, 1, 2, 3, 4, 5, 6, 7] , 5 ) --> 5
before __setitem__: [0, 1, 2, 3, 4, 5, 6, 7]
after  __setitem__( 3 , 33 ): [0, 1, 2, 33, 4, 5, 6, 7]
before __delitem__: [0, 1, 2, 33, 4, 5, 6, 7]
after  __delitem__( 3 ): [0, 1, 2, 4, 5, 6, 7]

I'm not terribly surprised by #7: append is probably implemented in an ad-hoc way. But for #4 and #6 I am confused. The __getitem__ documentation says: "Called to implement evaluation of self[key]. For sequence types, the accepted keys should be integers and slice objects." (my emphasys). And for __setitem__: " Same note as for __getitem__()", which I take to mean that key can also be a slice.

What's wrong with my reasoning? I'm prepared, if necessary, to override every list-modifying method (append, extend, insert, pop, etc.), but what should override to catch something like #6?

I am aware of the existence of __setslice__, etc. But those methods are deprecated since 2.0 ...

Hmmm. I read again the docs for __getslice__, __setslice__, etc., and I find this bone-chilling statement:

"(However, built-in types in CPython currently still implement __getslice__(). Therefore, you have to override it in derived classes when implementing slicing.)"

Is this the explanation? Is this saying "Well, the methods are deprecated, but in order to achieve the same functionality in 2.7.10 as you had in 2.0 you still have to override them"? Alas, then why did you deprecate them? How will things work in the future? Is there a "list" class - that I am not aware of - that I could extend and would not present this inconvenience? What do I really need to override to make sure I catch every list-modifying operation?

wim
  • 338,267
  • 99
  • 616
  • 750
Eduardo
  • 1,235
  • 16
  • 27
  • 2
    The deprecation of `__getslice__` and `__setslice__` was announced in the 2s, but not removed until Python 3. In Python 3, all indexing and slicing goes through `__XXXitem__`, where slicing passes `slice` objects (in Python 2, only extended slicing, including a step, goes to `__getitem__` ignoring `__getslice__`); `someobj[a:b]` is equivalent to `someobj.__getitem__(slice(a, b))` in Python 3, where in Python 2 it tries `someobj.__getslice__(a, b)` if it exists. – ShadowRanger Sep 25 '15 at 00:12

1 Answers1

5

The problem is that you're subclassing a builtin, and so have to deal with a few wrinkles. Before I delve into that issue, I'll go straight to the "modern" way:

How will things work in the future? Is there a "list" class - that I am not aware of - that I could extend and would not present this inconvenience?

Yes, there's the stdlib Abstract Base Classes. You can avoid the ugly complications caused by subclassing builtin list by using the ABCs instead. For something list-like, try subclassing MutableSequence:

from collections import MutableSequence

class MyList(MutableSequence):
    ...

Now you should only need to deal with __getitem__ and friends for slicing behaviour.

If you want to push ahead with subclassing the builtin list, read on...

Your guess is correct, you will need to override __getslice__ and __setslice__. The language reference explains why and you already saw that:

However, built-in types in CPython currently still implement __getslice__(). Therefore, you have to override it in derived classes when implementing slicing.

Note that l[3:7] will hook into __getslice__, whereas the otherwise equivalent l[3:7:] will hook into __getitem__, so you have to handle the possibility of receiving slices in both methods... groan!

wim
  • 338,267
  • 99
  • 616
  • 750