Python C API, send a python function pointer to c and execute it

Question

I want to create a function in python, pass it's function pointer to c and execute it there.

So my python file:

import ctypes
import example

def tester_print():
   print("Hello")

my_function_ptr = ctypes.CFUNCTYPE(None)(tester_print)
example.pass_func(my_function_ptr)

And here is what my function in c looks like:

typedef void (*MyFunctionType)(void);

PyObject* pass_func(PyObject *self, PyObject* args)
{
    PyObject* callable_object;
    if (!PyArg_ParseTuple(args, "O", &callable_object))
        return NULL;

    if (!PyCallable_Check(callable_object)) 
    {
        PyErr_SetString(PyExc_TypeError, "The object is not a callable function.");
        return NULL;
    }

    PyObject* function_pointer = PyCapsule_New(callable_object, "my_function_capsule", NULL);

    if (function_pointer == NULL) return NULL;

    MyFunctionType my_function = (MyFunctionType) PyCapsule_GetPointer(function_pointer, "my_function_capsule");

    if (my_function == NULL) return NULL;

    my_function(); // Or (*my_function)() Both same result.

    // PyCapsule_Free(function_pointer);

    Py_RETURN_NONE;
}

Doing this causes a seg fault on my_function() call. How can I do this?

Are you just trying to send a Python function? In that case, there's no need for ctypes function pointer shenanigans; just call the object via e.g. PyObject_Call. — nneonneo, Dec 23 '22 at 12:50
@nneonneo I'm going to do some operations with, but not limited to, calling the function and using it's pointer to set threads and such. — Turgut, Dec 23 '22 at 12:54
If you're attempting to start a thread using a Python function, I'd recommend creating a C function to start the thread and pass it the Python callable (most thread libraries let you specify an argument for the new thread). — nneonneo, Dec 23 '22 at 12:57
@nneonneo Yes that's exactly what I'm trying to do! Can you please embed that into my answer so I don't do something wrong again? — Turgut, Dec 23 '22 at 13:11

nneonneo · Accepted Answer · 2022-12-23T13:59:14.043

If you're just trying to pass a Python function to a C extension, pass it directly (don't use ctypes) and use PyObject_Call to call it:

example.pass_func(tester_print)

and

PyObject_CallNoArgs(callable_object);

If you need a real C function pointer for whatever reason, the usual approach is to write a C wrapper that takes the callable as an argument:

void callable_wrapper(PyObject *func) {
    PyObject_CallNoArgs(func);
    // plus whatever other code you need (e.g. reference counting, return value handling)
}

Most reasonable C APIs that take a callback function also provide a way to add an arbitrary argument to the callable ("user data"); for example, with pthreads:

result = pthread_create(&tid, &attr, callable_wrapper, callable_object);

Make sure to handle reference counting correctly: increment the reference on your callable object before passing it to the C API, and decrement the reference when it is no longer needed (e.g. if the callback is only called once, the callable_wrapper could DECREF before returning).

When using threads, you additionally need to ensure that you hold the GIL when calling any Python code; see https://docs.python.org/3/c-api/init.html#non-python-created-threads for more details and a code sample.

What your current code is doing is receiving a pointer to a ctypes CFUNCTYPE object as callable_object, placing that pointer in a capsule, taking it back out again, and calling it as if it was a C function pointer. This doesn't work, since it effectively attempts to call the CFUNCTYPE object as if it were a C function (the capsule stuff winds up being useless). When you're using the Python C API, there's almost never any need for ctypes in Python, because the C API can directly interact with Python objects.

Will this work on std::thread aswell? I've put the PyObject_CallNoArgs call inside a lambda and will attempt to do it that way. — Turgut, Dec 23 '22 at 13:21
Yes, you can pass the argument separately (e.g. `std::thread(callable_wrapper, callable_object)`), but of course there are many other ways to construct a suitable callable for std::thread in C++. — nneonneo, Dec 23 '22 at 13:26
You need to make sure that you hold the GIL inside your thread when you call your Python function. — DavidW, Dec 23 '22 at 13:56

Python C API, send a python function pointer to c and execute it

1 Answers1