Pass extra optional arguments to callbacks without breaking existing callbacks

Question

I have an API method that accepts a callback. The callback expects one argument.

I would like this method to pass a second argument to callbacks that accept it. However, I must maintain compatibility with callbacks that accept only the original argument. (In fact, I expect that most users will not care about the additional argument, so it would be annoying to force them to explicitly ignore it.)

I know that this can be done using inspect. I'm wondering if there is an "idiomatic" or commonly used solution that's not quite so heavyweight.

score 5 · Accepted Answer · answered Jan 09 '19 at 17:50

5

I think you can use __code__ to look how much arguments needed by the callback.

if callback.__code__.co_argcount == 2:
    callback(arg1, arg2)
else:
    callback(arg1)

This code isn't tested but it should work.

answered Jan 09 '19 at 17:50

Christoforus Surjoputro

558
6
17

Great solution. I forgot about the `__code__` attribute of a function object. – blhsing Jan 09 '19 at 17:54
Thanks @blhsing. Your try..except solution also works but with performace pinalty if exceptions called frequently. – Christoforus Surjoputro Jan 09 '19 at 18:00
Agreed. The `try-except` solution is much less efficient when the callback accepts only one parameter since it would add overhead in the creation of the exception and traceback objects. – blhsing Jan 09 '19 at 18:03
I didn't know about `__code__.co_argcount`. It looks like you could simplify/generalize it even further: `callback(*args[:callback.__code__.co_argcount])`. – Thom Smith Jan 09 '19 at 20:10
@thom-smith that's right if you pass argument through callback. With the "real" callback, you can't pass anything directly. The one who can pass argument is the caller. – Christoforus Surjoputro Jan 10 '19 at 01:46

blhsing · Answer 2 · 2019-01-09T18:01:45.433

2

A simpler solution would be to use a try block to try calling the callback with a second argument first, before falling back to calling with just one argument in the except block:

try:
    callback(first, second)
except TypeError as e:
    if e.__traceback__.tb_frame.f_code.co_name != 'func_name':
        raise
    callback(first)

edited Jan 09 '19 at 18:01

answered Jan 09 '19 at 16:43

blhsing

91,368
6
71
106

1

The trouble with this is that if `callback(first, second)` raises a `TypeError` for any other reason, then that error will be discarded and `callback` will be called a second time. – Thom Smith Jan 09 '19 at 17:10
True, though a well-written callback function should never raise a TypeError in the first place. But if that really is a concern, you can determine if the TypeError is caused by calling with the wrong number of arguments by checking the exception message as demonstrated in my updated answer). It does make the code look less elegant but it would work, and there is no reason to believe that the Python compiler is ever going to change its error messages for the existing built-in exceptions. – blhsing Jan 09 '19 at 17:34
I actually have an open issue where a test fails in 3.6+ because a built-in exception message changed. In addition, it's possible that `callback` may produce an indistinguishable `TypeError` with the same message. (These problems are admittedly unlikely to come up in practice.) – Thom Smith Jan 09 '19 at 17:40
It just occurred to me that a middle ground between this solution and the more complex solution in my answer would be to use `inspect.Signature.bind`. You could catch the `TypeError` before trying to run the callback. – Thom Smith Jan 09 '19 at 17:42
True. I've update my answer to use the traceback object instead. Substitute `func_name` with the name of the function this code is used. – blhsing Jan 09 '19 at 17:49

Thom Smith · Answer 3 · 2019-01-09T17:49:27.200

Using a function wrapper:

from inspect import signature, Parameter

def ignore_extra_arguments(function):
    positional_count = 0
    var_positional = False
    keyword_names = set()
    var_keyword = False

    for p in signature(function).parameters.values():
        if p.kind == Parameter.POSITIONAL_ONLY:
            positional_count += 1
        elif p.kind == Parameter.POSITIONAL_OR_KEYWORD:
            positional_count += 1
            keyword_names.add(p.name)
        elif p.kind == Parameter.VAR_POSITIONAL:
            var_positional = True
        elif p.kind == Parameter.KEYWORD_ONLY:
            keyword_names.add(p.name)
        elif p.kind == Parameter.VAR_KEYWORD:
            var_keyword = True

    if var_positional:
        new_args = lambda args: args
    else:
        new_args = lambda args: args[:positional_count]

    if var_keyword:
        new_kwargs = lambda kwargs: kwargs
    else:
        new_kwargs = lambda kwargs: {
            name: value for name, value in kwargs.items()
            if name in keyword_names
        }

    def wrapped(*args, **kwargs):
        return function(
            *new_args(args),
            **new_kwargs(kwargs)
        )

    return wrapped

It works, but it's a bit brute-force.

A simpler version, assuming that function has no keyword or variadic parameters:

from inspect import signature

def ignore_simple(function):
    count = len(signature(function).parameters)
    return lambda *args: function(*args[:count])

Pass extra optional arguments to callbacks without breaking existing callbacks

3 Answers3