Call cdef function by name in Cython

Question

I have a bunch of cdef functions in Cython, that are called by a def function in a pyx file, e.g.:

cdef inline void myfunc_c(...):
    (...)
    return

def wrapper(...):
    myfunc_c(...)
    return

This works well. But to simplify not having to have a python wrapper for each cdef function, I was trying to index the cdef functions by name, either by assigning them to a dictionary or something like:

def wrapper(operation):
    if operation == 'my_func':
        func = myfunc_c
    func(...)
    return

But this doesn't work. Cython complains that it doesn't know the type of myfunc_c.

Is there any way to index or call the cpdef functions by name (e.g. use a string)? I also tried things like locals()['myfunc_c'], but that doesn't work either.

I haven't used Cython before but this doesn't seem to make sense - why would you be able to call myfunc_c *inside* a Python function, but not *outside* one? — Jacob Garby, Sep 23 '17 at 10:50

DavidW · Accepted Answer · 2022-01-05T18:17:01.943

For a general cdef functions this is impossible - they only define a C interface but not a Python interface so there's no introspection available.

For a cdef function declared with api (e.g. cdef api funcname()) it is actually possible. There's an undocumented dictionary __pyx_capi__. This defines a dictionary (indexed by name) of PyCapsules containing function pointers. You'd then do

capsule_name = PyCapsule_GetName(obj)
func = PyCapsule_GetPointer(obj, capsule_name)

(where PyCapsule_* are functions cimported from the Python C API). func is a void* that you can cast into a function pointer of an appropriate type. Getting the type right is important, and up to you!

Although undocumented, the __pyx_capi__ interface is relatively stable and used by Scipy for its LowLevelCallable feature, for example.

cpdef functions define both a Python and a C interface. Within a function they will be available in globals() rather than locals() (since locals() only gives the variables defined in that function.)

I don't actually think you want to do this though. I think you just want to use cpdef instead of cdef since this automatically generates a Python wrapper for the function.

Maybe I should indeed go the `cpdef` way. I was avoiding it for performance reasons: I do need to do some pure python operations inside the "wrapper" part and was concerned that with `cpdef` it would not be fully optimised. — tiago, Sep 23 '17 at 11:08
It's probably worth timing it if you're worried. It's worth remembering that Cython does compile all of `def`, `cpdef` and `cdef` functions so the difference in speed is only from the function calling mechanism (and there won't be much difference between `cpdef` and `cdef`). — DavidW, Sep 23 '17 at 11:13
I've updated this with an alternative way that you can actually get `cdef` functions by name — DavidW, Jan 05 '22 at 18:17

Call cdef function by name in Cython

1 Answers1

Linked