6

I came across a peculiar behaviour of functools.update_wrapper: it overwrites the __dict__ of the wrapper object by that of the wrapped object - which may hinder its use when nesting decorators.

As a simple example, assume that we are writing a decorator class that caches data in memory and another decorator class that caches data to a file. The following example demonstrates this (I made the example brief and omitted all cacheing logic, but I hope that it demonstrates the question):

import functools

class cached:
    cache_type = 'memory'
    def __init__(self, fcn):
        super().__init__()
        self.fcn = fcn
        functools.update_wrapper(self, fcn, updated=())

    def __call__(self, *args):
        print("Retrieving from", type(self).cache_type)
        return self.fcn(*args)

class diskcached(cached):
    cache_type = 'disk'

@cached
@diskcached
def expensive_function(what):
    print("expensive_function working on", what)

expensive_function("Expensive Calculation")

This example works as intended - its output is 

Retrieving from memory
Retrieving from disk
expensive_function working on Expensive Calculation

However, it took me long to make this work - at first, I hat not included the 'updated=()' argument in the functools.update_wrapper call. But when this is left out, then nesting the decorators does not work - in this case, the output is

Retrieving from memory
expensive_function working on Expensive Calculation

I.e. the outer decorator directly calls the innermost wrapped function. The reason (which took me a while to understand) for this is that functools.update_wrapper updates the __dict__ attribute of the wrapper to the __dict__ attribute of the wrapped argument - which short-circuits the inner decorator, unless one adds the updated=() argument.

My question: is this behaviour intended - and why? (Python 3.7.1)

Andi Kleve
  • 135
  • 4
  • Well, looking at the [doc](https://docs.python.org/3.7/library/functools.html?highlight=update_wrapper#functools.update_wrapper), this behaviors is apparently intended, although why and whether the situation you mentioned was foreseen, I don't know... – Silmathoron Dec 03 '18 at 22:33
  • Dupe: [Attribute access on a decorated callable class](https://stackoverflow.com/q/50030408/674039) – wim Dec 03 '18 at 22:38
  • 2
    Yes it is intended, and yes it is bad (for the very reason you have found). stdlib `functools.wraps` is a crappy design, [this blog post](https://hynek.me/articles/decorators/) is a good read and pick one of the better implemented wrappers from there - boltons, wrapt, and decorator.py all work pretty well. – wim Dec 03 '18 at 22:44
  • @wim: I don't think any of those options are appropriate for class-based decorators like what this question is using. (`functools.update_wrapper` is questionably appropriate as is, but the options in that article have even more problems with this kind of thing.) – user2357112 Dec 04 '18 at 00:57
  • 1
    Beg to differ - decorator's function wrapper will crash something like `TypeError: You are decorating a non function`, which is the import-time failure I'd want to see here, preventing weird runtime behaviour. – wim Dec 04 '18 at 01:18
  • @wim: Sounds like you object to the existence of class-based decorators. I would consider the appropriate option here to be just not using any sort of `wraps`/`update_wrapper`/other "make this thing look like the underlying thing" tool, rather than using a tool that throws a TypeError. – user2357112 Dec 04 '18 at 17:04
  • Well spotted, @wim ! While definitely not an exact duplicate, it's useful to also have that related one. OP, thanks for asking specifically about this behaviour and offering a solution along the way ! – Ciprian Tomoiagă Dec 22 '20 at 08:49

1 Answers1

4

Making a wrapper function look like the function it wraps is the point of update_wrapper, and that includes __dict__ entries. It doesn't replace the __dict__; it calls update.

If update_wrapper didn't do this, then if one decorator set attributes on a function and another decorator wrapped the modified function:

@decorator_with_update_wrapper
@decorator_that_sets_attributes
def f(...):
    ...

the wrapper function wouldn't have the attributes set, rendering it incompatible with the code that looks for those attributes.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • Yes, and sorry: in my first sentence I wasn't precise - of course, it updates `__dict__` and does not replace it (as I also write further down). And I agree with your answer. But in my example, the `update` damaged the nesting of decorators. I assume that the `update` may damage nested decorators in many cases when the decorators are implemented via classes. Thus, I suspect that `update_wrapper` was primarily written in mind with decorators implemented as functions and not with decorators implemented as functions? – Andi Kleve Dec 04 '18 at 00:46
  • @AndiKleve: There's a reason it's called *func*tools, and the `update_wrapper` docs call the arguments functions. While it can be used for other callables, and the docs do say it can be used with other callables, the design was based around functions. (Also you wrote "as functions" twice.) – user2357112 Dec 04 '18 at 00:50