the fundamental differences of the way to overwrite getattr and setattr

Question

My hope is to make attributes case-insensitive. But overwriting __getattr__ and __setattr__ are somewhat different, as indicated by the following toy example:

class A(object):

    x = 10

    def __getattr__(self, attribute):
        return getattr(self, attribute.lower())

    ## following alternatives don't work ##

#    def __getattr__(self, attribute):
#        return self.__getattr__(attribute.lower()) 

#    def __getattr__(self, attribute):
#        return super().__getattr__(attribute.lower()) 

    def __setattr__(self, attribute, value):
        return super().__setattr__(attribute.lower(), value)

    ## following alternative doesn't work ##

#    def __setattr__(self, attribute, value):
#        return setattr(self, attribute.lower(), value)

a = A()
print(a.x) ## output is 10
a.X = 2
print(a.X) ## output is 2

I am confused by two points.

I assume getattr() is a syntactic sugar for __getattr__, but they behave differently.
Why does __setattr__ need to call super(), while __getattr__ doesn't?

`__getattr__` is called for any attribute that doesn't exist; you don't want to create an infinite loop by using `getattr()` on a non-existing attribute! — Martijn Pieters, Jul 10 '18 at 16:58
As a side note, you’re missing `__delattr__` here. And that also implies that you’re trying to learn how to do this without reading the Data Model docs on customizing attribute access, which explains a lot of your confusion. There’s not that much to it (although the interactions with things like descriptors, type objects, etc. can get complex, you don’t need to worry about that yet), and the docs explain it nicely, while it’s not the kind of thing where you can just guess how it might work and go from there. — abarnert, Jul 10 '18 at 17:27
[Here's a link](https://docs.python.org/3/reference/datamodel.html#customizing-attribute-access). Notice that it directly explains the "an intentional asymmetry between `__getattr__()` and `__setattr__()`" that you're asking about, and also explains how `__setattr__` should use a base class method, and so on. — abarnert, Jul 10 '18 at 17:35

Martijn Pieters · Accepted Answer · 2018-07-10T18:04:01.230

I assume getattr() is a syntactic sugar for __getattr__, but they behave differently.

That's because the assumption is incorrect. getattr() goes through the entire attribute lookup process, of which __getattr__ is only a part.

Attribute lookup first invokes a different hook, namely the __getattribute__ method, which by default performs the familiar search through the instance dict and class hierarchy. __getattr__ will be called only if the attribute hasn't been found by __getattribute__. From the __getattr__ documentation:

Called when the default attribute access fails with an AttributeError (either __getattribute__() raises an AttributeError because name is not an instance attribute or an attribute in the class tree for self; or __get__() of a name property raises AttributeError).

In other words, __getattr__ is an extra hook to access attributes that don't exist, and would otherwise raise AttributeError.

Also, functions like getattr() or len() are not syntactic sugar for a dunder method. They almost always do more work, with the dunder method a hook for that process to call. Sometimes there are multiple hooks involved, such as here, or when creating an instance of a class by calling the class. Sometimes the connection is fairly direct, such as in len(), but even in the simple cases there are additional checks being made that the hook itself is not responsible for.

Why does __setattr__ need to call super(), while __getattr__ doesn't?

__getattr__ is an optional hook. There is no default implementation, which is why super().__getattr__() doesn't work. __setattr__ is not optional, so object provides you with a default implementation.

Note that by using getattr() you created an infinite loop! instance.non_existing will call __getattribute__('non_existing') and then __getattr__('non_existing'), at which point you use getattr(..., 'non_existing') which calls __getattribute__() and then __getattr__, etc.

In this case, you should override __getattribute__ instead:

class A(object):
    x = 10

    def __getattribute__(self, attribute):
        return super().__getattribute__(attribute.lower())

    def __setattr__(self, attribute, value):
        return super().__setattr__(attribute.lower(), value)

It might also be worth explaining that `spam(x)` is _never_ syntactic sugar for `x.__spam__()`, rather than just in the case of `__getattr__`. For example, `bool(x)` tries `x.__bool__` then `x.__len__`; `iter(x)` tries `x.__iter__` then `__len__` and `__getitem__`; `x+y` tries `x.__add__(y)` and `y.__radd__(x)` (in order depending on the type relationship); etc. Calling the dunder methods directly is not something you want to do except in specific special cases (like calling your super's `__setattr__`). — abarnert, Jul 10 '18 at 17:38
@abarnert: yes, and even in the simple cases such as `len()` calling `__len__` there are type and boundary checks you don't necessarily want to bypass. — Martijn Pieters, Jul 10 '18 at 18:04
I'm sure you remember the question from the guy who wanted to call `__len__` as an optimization, to avoid all that wasted time converting `AttributeError` to `TypeError`, and insisted `timeit` must be broken because it said his code was slower instead of faster, and refused to accept that it takes time to look up, build, and call a bound method… — abarnert, Jul 10 '18 at 18:16

the fundamental differences of the way to overwrite getattr and setattr

1 Answers1