creating a decorator that combines two functions without specifying the calling signature of the original function

Question

I want to create a decorator that combines two functions and combines the parameters from their signatures.

The interface I want:

def f(a, b, c, d, e, f, g, h, i, j, k, l, m, n):
    # I am using many parameters to explain the need of not
    # needing to type the arguments again.
    return a * b * c * d * e * f * g * h * i * j * k * l * m * n

@combines(f)
def g(o, p, *args, **kwargs):
    return (o + p) * f(*args, **kwargs)

This should essentially result in:

def g(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p):
    return (o + p) * (a * b * c * d * e * f * g 
                      * h * i * j * k * l * m * n)

The reason I want this is because I don't really know the arguments of function f (I know them, but I don't want to type them again in order to make it general.)

I am not sure if I have to call g with *args and **kwargs, but I think this will be necessary.

This is how far I got:

import functools
import inspect

def combines(old_func):
    old_sig = inspect.signature(old_func)
    old_parameters = old_sig.parameters
    def insert_in_signature(new_func):
        new_parameters = inspect.signature(new_func).parameters
        for new_parameter in new_parameters:
            if new_parameter in old_parameters.keys():
                raise TypeError('`{}` argument already defined'.format(new_parameter))

        @functools.wraps(new_func)
        def wrapper(*args, **kwargs):
            return old_func(*args, **kwargs) * new_func(*args, **kwargs)

        parms = list(old_parameters.values())
        for arg, par in new_parameters.items():
            if par.kind == inspect.Parameter.POSITIONAL_OR_KEYWORD:
                parms.append(inspect.Parameter(arg, par.kind))

        wrapper.__signature__ = old_sig.replace(parameters=parms)
        return wrapper
    return insert_in_signature

def f(a, b, c, d, e, f, g, h, i, j, k, l, m, n):
    return a * b * c * d * e * f * g * h * i * j * k * l * m * n

@combines(f)
def g(o, p, *args, **kwargs):
    return (o + p) * f(*args, **kwargs)

This results in the desired calling signature of g, but it does not work.

EDIT because the error message was asked

For example: g(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-23-2775f64e1b3e> in <module>()
----> 1 g(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1)

<ipython-input-18-3a843320e4e3> in wrapper(*args, **kwargs)
     13         @functools.wraps(new_func)
     14         def wrapper(*args, **kwargs):
---> 15             return old_func(*args, **kwargs) * new_func(*args, **kwargs)
     16 
     17         parms = list(old_parameters.values())

TypeError: f() takes 14 positional arguments but 16 were given

if I then follow the error message and give 14 arguments with g(1,1,1,1,1,1,1,1,1,1,1,1,1,1):

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-24-052802b037a4> in <module>()
----> 1 g(1,1,1,1,1,1,1,1,1,1,1,1,1,1)

<ipython-input-18-3a843320e4e3> in wrapper(*args, **kwargs)
     13         @functools.wraps(new_func)
     14         def wrapper(*args, **kwargs):
---> 15             return old_func(*args, **kwargs) * new_func(*args, **kwargs)
     16 
     17         parms = list(old_parameters.values())

<ipython-input-18-3a843320e4e3> in g(o, p, *args, **kwargs)
     29 @combines(f)
     30 def g(o, p, *args, **kwargs):
---> 31     return (o + p) * f(*args, **kwargs)

TypeError: f() missing 2 required positional arguments: 'm' and 'n'

So clearly my implementation is not really working.

In what specific way does it not work? Do you have a _complete_ example other people can try? Show it. Is there an error? Show _that_. — Useless, Feb 01 '17 at 15:13

score 2 · Accepted Answer · answered Feb 02 '17 at 09:55

Your problem is that you call the f function twice with different parameters, once inside the original g with only its expected parameters and once inside the wrapper with all the parameters.

You must choose one, my advice is to remove its call from the original g

I have slightly changed your code, but at least my version works in Python 3.5:

the signature of the wrapped function lists all parameter for f and for g
the wrapped function accepts positional and keyword parameters
the wrapped function raises an error when it receives an incorrect number of parameters

Here is the code:

def combine(ext):
    ext_params = inspect.signature(ext).parameters
    def wrapper(inn):
        inn_params = inspect.signature(inn).parameters
        for k in inn_params.keys():
            if k in ext_params.keys():
                raise TypeError('`{}` argument already defined'.format(
                        k))
        all_params = list(ext_params.values()) + \
                 list(inn_params.values())
        # computes the signature for the wrapped function
        sig = inspect.signature(inn).replace(parameters = all_params)
        def wrapped(*args, **kwargs):
            # signature bind magically processes positional and keyword arguments
            act_args = sig.bind(*args, **kwargs).args  
            ext_args = act_args[:len(ext_params.keys())] #  for external function
            inn_args = act_args[len(ext_params.keys()):] #  for inner function
            return ext(*ext_args) * inn(*inn_args)
        w = functools.update_wrapper(wrapped, inn) # configure the wrapper function
        w.__signature__ = sig   # and set its signature
        return w
    return wrapper

I can now write:

>>> @combine(f)
def g(o,p):
    return o+p

>>> help(g)
Help on function g in module __main__:

g(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p)

>>> g(1,1,1,1,1,1,1,1,1,1,1,1,1,p=1, o=1, n=1)
2

score 0 · Answer 2 · answered Feb 01 '17 at 16:07

I think I understand what you want. For that, first of all, I recommend using OOP, to make things clearer. I don't think you need all that code to proccess the parameters or the inspect module either. I think you should keep it simple:

I think part of the problem is that you call both functions with the same parameters and they're considered too many arguments, or too few, in which case you would need to modify g. I guess there is no way to figure out which parameters go to which function so here are a few solutions:

Here is one solution:

def combines(func):
    class Decorator:
        def __init__(self, f):
            self.outer_func = func
            self.inner_func = f

        def __call__(self, *args, **kwargs):
            res, a, p = self.inner_func(*args, **kwargs)
            return res * self.outer_func(*a, **p)
    return Decorator


def f(a, b, c, d, e, f, g, h, i, j, k, l, m, n):
    return a * b * c * d * e * f * g * h * i * j * k * l * m * n


@combines(f)
def g(o, p, *args, **kwargs):
   return o+p, args, kwargs

print g(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1) # output: 2

Here is another solution which uses a third function to combine the two:

def combines(func1, func2):
    class Decorator:
        def __init__(self, f):
            self.outer_func = func1
            self.inner_func = func2
            self.func = f

        def __call__(self, *args, **kwargs):
            res, a, p = self.inner_func(*args, **kwargs)
            return self.func(res, self.outer_func(*a, **p))
    return Decorator


def f(a, b, c, d, e, f, g, h, i, j, k, l, m, n):
    return a * b * c * d * e * f * g * h * i * j * k * l * m * n


def g(o, p, *args, **kwargs):
    return o+p, args, kwargs


@combines(f, g)
def h(a, b):        # a is g's result
    return a * b    # b is f's result

print h(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)     # output: 2

To write a function to figure out which parameters will go to which function would be hard, if not impossible, but I think this should work.

creating a decorator that combines two functions without specifying the calling signature of the original function

2 Answers2