Python nonlocal variable value retained after multiple decorator calls

Question

Sorry for the disturbing title, but I can't put the right words on what I've been looking at for hours now.

I'm using a decorator with optional parameters and i want to alter one of them within the wrapped function so that each call ends up doing a different thing. For context (that I had to remove), i want to create some sort of hash of the original function args and work with that.

from functools import  wraps


def deco(s=None):
    def _decorate(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            print(locals())
            
            nonlocal s  # Get the deco func arg
            if not s:
                s = "some_value_depending_on_other_args_i_removed"
            
            # do_smth_smart(s)           

            # s=None  # This solves my issue if uncommented
            return None

        return wrapper
    return _decorate

Here is a small test sample :

@deco()
def test1(self,x):
    pass
    
@deco()
def test2(self,a,b):
    pass

test1(1)
test1(2)

test2(3,4)
test2(5,6)

I would expect s to be "reset" to None whenever I call the decorated functions.

To my surprise, as it stands, the output is :

{'args': (1,), 'kwargs': {}, 's': None}
{'args': (2,), 'kwargs': {}, 's': 'newname'}
{'args': (3, 4), 'kwargs': {}, 's': None}
{'args': (5, 6), 'kwargs': {}, 's': 'newname'}

Would someone enlighten me please ? Thanks in advance :)

"I would expect s to be "reset" to None whenever I call the decorated functions." Why would it be reset? It's hard to give a good explanation without understanding your thinking — juanpa.arrivillaga, May 12 '23 at 17:34
either accept mine or @juanpa.arrivillaga answers or provide us with some feedback so we can edit our answers. — Ori Yarden PhD, Jul 11 '23 at 20:04

Ori Yarden PhD · Accepted Answer · 2023-05-13T00:05:54.900

If we add some print statements:

from functools import  wraps

def deco(s=None):
    print('deco')
    def _decorate(func):
        print('_decorate')
        @wraps(func)
        def wrapper(*args, **kwargs):
            print('wrapper')
            print(locals())
            
            nonlocal s  # Get the deco func arg
            if not s:
                s = "some_value_depending_on_other_args_i_removed"
            
            # do_smth_smart(s)           

            # s=None  # This solves my issue if uncommented
            return None

        return wrapper
    return _decorate

@deco()
def test1(self,x):
    pass
    
@deco()
def test2(self,a,b):
    pass

test1(1)
test1(2)

test2(3,4)
test2(5,6)

We can see that deco (and _decorate) is only called when decorating the functions test1 and test2:

deco
_decorate
deco
_decorate
wrapper
{'args': (1,), 'kwargs': {}, 's': None}
wrapper
{'args': (2,), 'kwargs': {}, 's': 'some_value_depending_on_other_args_i_removed'}
wrapper
{'args': (3, 4), 'kwargs': {}, 's': None}
wrapper
{'args': (5, 6), 'kwargs': {}, 's': 'some_value_depending_on_other_args_i_removed'}

Even here only wrapper is really called every time along with the function it's wrapping, so that's where s needs to be; to achieve what you're asking we can do something like assigning _s's __default__ value to be s in wrapper's parameter definition:

from functools import  wraps

def deco(s=None):
    print('deco')
    def _decorate(func):
        print('_decorate')
        @wraps(func)
        def wrapper(*args, _s=s, **kwargs):
            print('wrapper')
            print(locals())
            
            nonlocal s  # Get the deco func arg
            if not _s:
                _s = "some_value_depending_on_other_args_i_removed"
            print(locals())
            # do_smth_smart(s)           

            # s=None  # This solves my issue if uncommented
            return None

        return wrapper
    return _decorate

@deco(1)
def test1(self,x):
    pass
    
@deco()
def test2(self,a,b):
    pass

test1(1)
test1(2)

test2(3,4)
test2(5,6)

Outputs:

deco
_decorate
deco
_decorate
wrapper
{'_s': 1, 'args': (1,), 'kwargs': {}, 's': 1}
{'_s': 1, 'args': (1,), 'kwargs': {}, 's': 1}
wrapper
{'_s': 1, 'args': (2,), 'kwargs': {}, 's': 1}
{'_s': 1, 'args': (2,), 'kwargs': {}, 's': 1}
wrapper
{'_s': None, 'args': (3, 4), 'kwargs': {}, 's': None}
{'_s': 'some_value_depending_on_other_args_i_removed', 'args': (3, 4), 'kwargs': {}, 's': None}
wrapper
{'_s': None, 'args': (5, 6), 'kwargs': {}, 's': None}
{'_s': 'some_value_depending_on_other_args_i_removed', 'args': (5, 6), 'kwargs': {}, 's': None}

you need the extra level of wrapping to pass arguments to the decorator — juanpa.arrivillaga, May 12 '23 at 17:37
@juanpa.arrivillaga which would be those in `wrapper`, unless you mean that OP wants to pass in a value for `s` in the decorator (else `None`)? — Ori Yarden PhD, May 12 '23 at 17:41
Yes, the OP wants to be able to do something like `@deco(s=42)....` — juanpa.arrivillaga, May 12 '23 at 17:43

score 0 · Answer 2 · answered May 12 '23 at 17:42

It is unclear why you expected the variable to be "reset". That is not how free variables work in Python. Consider a much simpler example:

>>> def outer(x):
...     def inner():
...         nonlocal x
...         print(x)
...         x += 1
...     return inner
...
>>> f = outer(8)
>>> f()
8
>>> f()
9
>>> f()
10
>>> f = outer(8)
>>> f()
8
>>> f()
9
>>> f()
10

And semantically it is no different than modifying a global variable in your function:

>>> x = 1
>>> def foo():
...     global x
...     print(x)
...     x += 1
...
>>> foo()
1
>>> foo()
2
>>> foo()
3
>>> print(x)
4

Python nonlocal variable value retained after multiple decorator calls

2 Answers2