3

I am trying to call a C-function from Python that takes a function pointer as parameter. I need that function to be dynamically determined at runtime. Using ctypes, that is pretty simple.

The C-code could look like this:

// my_c_funcs.h
double mul(double, double);
double add(double, double);
double do_something(double (*)(double, double), double, double);

// my_c_funcs.h
int mul(int a, int b) {
    return a*b;
}
int add(int a, int b) {
    return a + b;
}
int do_something(int (*f)(int, int), int a, int b) {
    return f(a, b);
}

Having compiled that code into a shared library named "libMyCFuncs.so", I can pass a function that is determined at runtime in Python using ctypes:

# foo.py
import ctypes

lib = ctypes.cdll.LoadLibrary("./libMyCfuncs.so")

def foo(func_name, a, b):
    func = getattr(lib, func_name)
    return lib.do_something(func, a, b)

I know I should define the return types but I left that out for brevity and used ints only.

The code above gives the expected result, e.g. calling foo.foo('add', 2, 4) yields 6. However, I'd prefer using Cython because I make a lot of use of two- or higher-dimensional arrays and passing arrays is IMHO a lot easier in Cython. Let's say the Cython code is in "foo.pyx":

# foo.pyx
cdef extern from "my_c_funcs.h":
    int mul(int, int)
    int add(int, int)
    int do_something(int (*)(int, int), int, int)

def foo(func_name, int a, int b):
    # ???

Calling getattr or even eval obviously doesn't work. So how can I realize that in Cython?

BernieD
  • 352
  • 1
  • 10
  • Honestly, this sounds quite similar to scipy's [LowLevelCallable](https://github.com/scipy/scipy/blob/v1.7.0/scipy/_lib/_ccallback.py) class. It basically acts as a wrapper around a PyCapsule object (which in turn is a python object that wraps a C pointer). There are some limits to the `from_cython` method of that class though, since an api is not exported from cdef class c-level functions, but otherwise for wrapping raw C-library or module level cdef cython function, the approach outlined in that code should work. – CodeSurgeon Jul 03 '21 at 23:54
  • Very similar: https://stackoverflow.com/q/55918787/5769463 – ead Jul 04 '21 at 04:03
  • @CodeSurgeon That seems interesting. I'm going to have a look at that. – BernieD Jul 05 '21 at 14:19
  • @ead I thought about that solution. It works for the the example. But the actual application I am working on requires a variable 'do_something'-function as well. And there are quite a lot combinations, which would result in a large number of ```elif```s. Moreover, the code would be hardly extensible. – BernieD Jul 05 '21 at 14:23

1 Answers1

1

You have to provide a wrapper from cython that can be called in your shared object. Cython essentially has three "modes"

  1. def : normal python functions
  2. cpdef: python callable function with possible c/cpp internal variables
  3. cdef: pure c/cpp function

As an example for your code a simple binding would be


cdef extern from "c_funcs.h":
    double mul(double, double)
    double add (double, double)
    double do_something(double(*)(double, double), double, double)

cdef extern from "c_funcs.c":
    pass

# this is callable from python
cpdef double py_do_something_mul(str name, double x,  double y):
    return do_something(mul, x, y)

If you want something like dynamic dispatching, you would have to provide some wrapping to this as well. It won't work with the default python dicts, but an ordered or unorered_map would work for this.

cvanelteren
  • 1,633
  • 9
  • 16