2

Python decorators are a very "pythonic" solution to a lot of problems. Because of this, I'd like to include a pre-defined decorator in my C-extension that can decorate functions that are called in Python files that include my extension.

I can't seem to find anything in the CPython api documentation that describes how write decorators. Any help/direction would be appreciated.

Specifically, I'd like the Python code to look like the following:

import my_c_extension as m

@m.my_decorator(1)
def func():
    pass


func()

Where my_decorator would use the argument '1' to carry out some functionality (written in c inside my_c_extension) before func is called, and more functionality after func is called.

Thanks in advance!

jasbury
  • 35
  • 1
  • 4
  • Have you tried writing a decorator in python and then using cytgon to translate it to C and looking at the output? – plugwash Oct 13 '21 at 08:05
  • There isn't anything particularly special about decorators - they're just functions that take functions as arguments and return functions. – DavidW Oct 13 '21 at 08:21
  • @plugwash good catch, that's definitely one solution I've tried! But generated Cython code is not always simple/readable as a one-time from-scratch solution might be. I would have used Cython itself, but the module uses many low level hardware interfaces. – jasbury Oct 13 '21 at 20:18
  • @DavidW It seems easy at first, but it gets confusing with translating between C functions and Python function-objects as well as Python vs C scoping rules. As with the example below by Mr Fooz, 'Wrapped' has access to func from the outer function which is hard to emulate in C. At least using the Python C-APIs that I know of off hand... – jasbury Oct 13 '21 at 20:22
  • Yeah - agreed - emulating closure scopes is fairly hard in C (I did actually [write an answer on it fairly recently](https://stackoverflow.com/questions/69030621/how-to-implement-closure-in-python-using-c-api/69035336#69035336)). It's undoubtedly tricky to implement a good decorator but there's nothing particularly specific about doing it. I'd give serious thought to using Cython - you can of course write your low level bits in C and just have Cython call those C functions – DavidW Oct 13 '21 at 20:50

1 Answers1

0

For problems like this, it's often helpful to first mock things up in Python. In your case, I expected there'd be two benefits:

  • If you do end up implementing the decorator entirely in an extension module, this will help you understand how it should work and what state will be held by which objects you'll be creating.

  • Often, you can simplify the problem by doing most of the work in Python.

For the sake of argument, let's say what you're really trying to do is use a C extension to help sample low-level CPU counters for performance tuning. Here would be a way to achieve that with a hybrid of Python and C. All the complicated decorator magic stays in Python while only the functionality that actually needs to be in C is in C.

def print_cpu_counter_info(counter_id):
    def wrapper(func):
        def wrapped(*args, **kwargs):
            before = my_c_extension.get_counter_value(counter_id)
            ret = func(*args, **kwargs)
            after = my_c_extension.get_counter_value(counter_id)
            print(f'counter {counter_id}: {before} -> {after}')
            return ret
        return wrapped
    return wrapper

@print_cpu_counter_info(L3_CACHE_LINE_MISS_COUNTER_ID)
def my_slow_func(...):
    ...

If that's not viable (too slow, more complicated than this, etc.), then you'll need to create extension objects in C that replicate the behavior of the wrapper and wrapped functions. It's definitely doable, but it'll take a bit of work, and it'll be harder to maintain. Expect to write hundreds to thousands of lines of C code to replicate what only took a few lines of Python code in the above example.

Mr Fooz
  • 109,094
  • 6
  • 73
  • 101