How do classes inherited from namedtuple maintain access to parent properties with a redefined init?

Question

Minimum Reproducible Example:

from collections import namedtuple

class Test(namedtuple('Test', ['a','b'])):
           def __init__(self, a, b):
                self.c = self.a + self.b
           def __str__(self):
                return self.c

print(Test('FIRST', 'SECOND'))

OUTPUT:

FIRSTSECOND

I thought when an __init__ function is defined, it overwrites the parent implementation. If that is the case, how do self.a and self.b exist with the correct values? If I forego the a and b parameters in __init__, I get a TypeError: __init__() takes 1 positional argument but 3 were given. I need to provide the parameters, but they're not being set explicitly in __init__ either, and I have no called to super().

Because `self.a` and `self.b` are not attributes. The underlying `tuple` structure is created in `tuple.__new__`. Honestly, what you have above shouldn't be a named tuple to begin with, maybe just a dataclass if you want to avoid boilerplate? Because the above defeats the space-saving advantage of `namedtuple` entirely — juanpa.arrivillaga, Dec 06 '20 at 17:47
I want an immutable data type with a custom defined `__str__`. I thought subclassing from `namedtuple` would be the best way to do that — ajoseps, Dec 06 '20 at 17:49
`namedtuple` isn't a class, you cannot subclass from it. it is a *class factory*. `namedtuple('Test', ['a','b'])` returns a type, a `tuple` subclass. But the way you've done it here you've made your namedtuple mutable to boot! (note, it has to be, since you are doing `self.c = self.a + self.b`. Again, perhaps consider a `dataclasses.dataclass` — juanpa.arrivillaga, Dec 06 '20 at 17:50
@juanpa.arrivillaga I don't think the class is fully mutable. currently the `c` attribute is mutable and the `a` and `b` attributes are still immutable. I would like `c` to also be immutable though. I'll take a look at `dataclasses.dataclass` thank you. — ajoseps, Dec 06 '20 at 17:53
Right. "not fully mutable" to me still means "mutable". In any case, chasing strict immutability in Python, aside from using a basic `namedtuple` is generally not worth the effort and can practivally always be easily subverted unless you write a C-extension — juanpa.arrivillaga, Dec 06 '20 at 17:54

chepner · Accepted Answer · 2020-12-06T18:07:52.733

self.a and self.b are set by the named tuple's __new__ method before __init__ is called. This is because a named tuple is immutable (aside from the ability to add additional attributes, as Test.__init__ does), so trying to set a and b after the tuple is created would fail. Instead, the values are passed to __new__ so that the values are available when the tuple is being created.

Here's an example of __new__ being overriden to swap the a and b values.

class Test(namedtuple('Test', ['a','b'])):
    def __new__(cls, a, b, **kwargs):
        return super().__new__(cls, b, a, **kwargs)

    def __init__(self, a, b):
        self.c = self.a + self.b

    def __str__(self):
        return self.c

print(Test('FIRST', 'SECOND'))  # outputs SECONDFIRST

Trying to do the same with __init__ would fail:

class Test(namedtuple('Test', ['a','b'])):
    def __init__(self, a, b):
        self.a, self.b = b, a
        self.c = self.a + self.b

    def __str__(self):
        return self.c

print(Test('FIRST', 'SECOND'))  # outputs SECONDFIRST

results in

Traceback (most recent call last):
  File "/Users/chepner/advent-of-code-2020/tmp.py", line 11, in <module>
    print(Test('FIRST', 'SECOND'))
  File "/Users/chepner/advent-of-code-2020/tmp.py", line 5, in __init__
    self.a, self.b = b, a
AttributeError: can't set attribute

To make c immutable as well (while keeping it distinct from the tuple itself), use a property.

class Test(namedtuple('Test', ['a','b'])):
    @property
    def c(self):
        return self.a + self.b

    def __str__(self):
        return self.c

Note that c is not visible or accessible when treating an instance of Test as a regular tuple:

>>> x = Test("First", "Second")
>>> x
Test(a='First', b='Second')
>>> len(x)
2
>>> tuple(x)
('First', 'Second')
>>> x[0]
'First'
>>> x[1]
'Second'
>>> x[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: tuple index out of range
>>> x.c = "foo"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: can't set attribute

If I wanted `c` to also be immutable, would that be something i set in `__new__` and forego defining an `__init__`? — ajoseps, Dec 06 '20 at 17:51
@ajoseps you would need to use `namedtuple('Test', ['a','b','c'])` then, and maybe in `__new__` just do something like `return super().__new__(cls, a, b, a+b)` — juanpa.arrivillaga, Dec 06 '20 at 17:52
No; the type returned by `namedtuple` is a subclass of `tuple`; the attributes defined by it are "special", not because they are initialized via `__new__`, but because of how the type is defined in the first place. If you want `c` to be read-only, your best bet would be to make it a property without a setter. — chepner, Dec 06 '20 at 17:52
(I can imagine a case where you want to distinguish between values that are part of the tuple, and values that are attributes of the tuple, even if a named tuple allows access to both using the same syntax.) — chepner, Dec 06 '20 at 17:53
yeah I want to be able to specify an instance of the datatype with 2 parameters, and have a 3rd property be derived from the 2, but immutable. I would want to avoid explicitly specifying 3 properties in a `namedtuple` since that might lead to the third being explicitly specified. — ajoseps, Dec 06 '20 at 17:55

How do classes inherited from namedtuple maintain access to parent properties with a redefined __init__?

1 Answers1

How do classes inherited from namedtuple maintain access to parent properties with a redefined init?