Custom list class in Python 3 with __get__ and __set__ attributes

Question

I would like to write a custom list class in Python 3 like in this question How would I create a custom list class in python?, but unlike that question I would like to implement __get__ and __set__ methods. Although my class is similar to the list, but there are some magic operations hidden behind these methods. And so I would like to work with this variable like with list, like in main of my program (see below). I would like to know, how to move __get__ and __set__ methods (fget and fset respectively) from Foo class to MyList class to have only one class.

My current solution (also, I added output for each operation for clarity):

class MyList:

    def __init__(self, data=[]):
        print('MyList.__init__')
        self._mylist = data 

    def __getitem__(self, key):
        print('MyList.__getitem__')
        return self._mylist[key]

    def __setitem__(self, key, item):
        print('MyList.__setitem__')
        self._mylist[key] = item

    def __str__(self):
        print('MyList.__str__')
        return str(self._mylist)


class Foo:

    def __init__(self, mylist=[]):
        self._mylist = MyList(mylist)

    def fget(self):
        print('Foo.fget')
        return self._mylist

    def fset(self, data):
        print('Foo.fset')
        self._mylist = MyList(data)

    mylist = property(fget, fset, None, 'MyList property')


if __name__ == '__main__':
    foo = Foo([1, 2, 3])
    # >>> MyList.__init__
    print(foo.mylist)
    # >>> Foo.fget
    # >>> MyList.__str__
    # >>> [1, 2, 3]
    foo.mylist = [1, 2, 3, 4]
    # >>> Foo.fset
    # >>> MyList.__init__
    print(foo.mylist)
    # >>> Foo.fget
    # >>> MyList.__str__
    # >>> [1, 2, 3, 4]
    foo.mylist[0] = 0
    # >>> Foo.fget
    # >>> MyList.__setitem__
    print(foo.mylist[0])
    # >>> Foo.fget
    # >>> MyList.__getitem__
    # >>> 0

Thank you in advance for any help.

---

How to move __get__ and __set__ methods (fget and fset respectively) from Foo class to MyList class to have only one class?

---

UPD:

Thanks a lot to @Blckknght! I tried to understand his answer and it works very well for me! It's exactly what I needed. As a result, I get the following code:

class MyList:
    def __init__(self, value=None):
        self.name = None
        if value is None:
            self.value = []
        else:
            self.value = value

    def __set_name__(self, owner, name):
        self.name = "_" + name

    def __get__(self, instance, owner):
        return getattr(instance, self.name)

    def __set__(self, instance, value):
        setattr(instance, self.name, MyList(value))

    def __getitem__(self, key):
        return self.value[key]

    def __setitem__(self, key, value):
        self.value[key] = value

    def append(self, value):
        self.value.append(value)

    def __str__(self):
        return str(self.value)


class Foo:
    my_list = MyList()

    def __init__(self):
        self.my_list = [1, 2, 3]
        print(type(self.my_list))       # <class '__main__.MyList'>
        self.my_list = [4, 5, 6, 7, 8]
        print(type(self.my_list))       # <class '__main__.MyList'>
        self.my_list[0] = 10
        print(type(self.my_list))       # <class '__main__.MyList'>
        self.my_list.append(7)
        print(type(self.my_list))       # <class '__main__.MyList'>
        print(self.my_list)             # [10, 5, 6, 7, 8, 7]

foo = Foo()

I don't know, that's Pythonic way or not, but it works as I expected.

Answer 1

In a comment, you explained what you actually want:

x = MyList([1])
x = [2]
# and have x be a MyList after that.

That is not possible. In Python, plain assignment to a bare name (e.g., x = ..., in contrast to x.blah = ... or x[0] = ...) is an operation on the name only, not the value, so there is no way for any object to hook into the name-binding process. An assignment like x = [2] works the same way no matter what the value of x is (and indeed works the same way regardless of whether x already has a value or whether this is the first value being assigned to x).

Answer 2

While you can make your MyList class follow the descriptor protocol (which is what the __get__ and __set__ methods are for), you probably don't want to. That's because, to be useful, a descriptor must be placed as an attribute of a class, not as an attribute of an instance. The properties in your Foo class creating separate instances of MyList for each instance. That wouldn't work if the list was defined on the Foo class directly.

That's not to say that custom descriptors can't be useful. The property you're using in your Foo class is a descriptor. If you wanted to, you could write your own MyListAttr descriptor that does the same thing.

class MyListAttr(object):
    def __init__(self):
        self.name = None

    def __set_name__(self, owner, name):     # this is used in Pyton 3.6+
        self.name = "_" + name

    def find_name(self, cls):  # this is used on earlier versions that don't support set_name
        for name in dir(cls):
            if getattr(cls, name) is self:
                self.name = "_" + name
                return
        raise TypeError()

    def __get__(self, obj, owner):
        if obj is None:
           return self
        if self.name is None:
            self.find_name(owner)
        return getattr(obj, self.name)

    def __set__(self, obj, value):
        if self.name is None:
            self.find_name(type(obj))
        setattr(obj, self.name, MyList(value))

class Foo(object):
    mylist = MyListAttr() # create the descriptor as a class variable

    def __init__(self, data=None):
        if data is None:
            data = []

        self.mylist = data    # this invokes the __set__ method of the descriptor!

The MyListAttr class is more complicated than it otherwise might be because I try to have the descriptor object find its own name. That's not easy to figure out in older versions of Python. Starting with Python 3.6, it's much easier (because the __set_name__ method will be called on the descriptor when it is assigned as a class variable). A lot of the code in the class could be removed if you only needed to support Python 3.6 and later (you wouldn't need find_name or any of the code that calls it in __get__ and __set__).

It might not seem worth writing a long descriptor class like MyListAttr to do what you were able to do with less code using a property. That's probably correct if you only have one place you want to use the descriptor. But if you may have many classes (or many attributes within a single class) where you want the same special behavior, you will benefit from packing the behavior into a descriptor rather than writing a lot of very similar property getter and setter methods.

You might not have noticed, but I also made a change to the Foo class that is not directly related to the descriptor use. The change is to the default value for data. Using a mutable object like a list as a default argument is usually a very bad idea, as that same object will be shared by all calls to the function without an argument (so all Foo instances not initialized with data would share the same list). It's better to use a sentinel value (like None) and replace the sentinel with what you really want (a new empty list in this case). You probably should fix this issue in your MyList.__init__ method too.