Cython: How to make an array of functions with different signatures

Question

In Cython I would like to store an array of functions, but the functions don't all have the same signature. Specifically, some have two parameters and some three.

I defined the following:

ctypedef long (*func2param)(long param1, long param2)
ctypedef long (*funct3param)(long param1, long param2, long param3)

However, even if I had just one such signature, I'm still not sure how to go about making it work. Trying to assign a cdef function to an array of one of the above types gives me:

func_array2[i] = func_list[i][FUNCTION]

Cannot convert Python object to 'func2param'

Trying to cast gives:

func_array2[i] = <func2param>func_list[i][FUNCTION]

Python objects cannot be cast to pointers of primitive types

Do you want to know why your code doesn't build - then what are `func_list` and `func_array2`? Or do you want to know "How to make an array of functions with different signatures?" - there are some ways to achieve that (the easiest to cast everything to a `uintptr_t`) but depending on what you want to do with those pointers in array not all will be helpful to you - also here more informations is needed. There is also question, whether you should be doing, what you'are trying to do... — ead, Aug 13 '19 at 04:47
Yes, I want to know "How to make an array of functions with different signatures?" As per my explanation above, I have two signatures and I want an array that can store both types of functions. The examples of why it doesn't build was just my attempts to try to do what made intuitive sense to me and the errors I got back. — Bruce Nielson, Aug 21 '19 at 18:54

DavidW · Answer 1 · 2019-08-14T12:45:59.077

I can't think of anything useful you can do with an array of function pointers of unknown type - it isn't safe to call them (since you don't know the signature) and there really isn't anything else to do with function pointers. Therefore you at least need to find some way of storing what type of pointer it is.

One option would be to store a struct containing both pointers:

cdef struct FuncPtrStruct:
    func2param f2p
    func3param f3p

You then set one to NULL, store in the other one, and only call the non-NULL one The array specification would then be similar to cdef FuncPtrStruct array[10].

However, I'd probably use a C Union instead to store both pointers in the same memory location (+ an enum to identify the type). This is a little more effort to set up (you need an enum to define the type of the union, the union itself, and a struct containing the enum and the union); however the advantage is that you can add a lot more different types of function pointers without using more memory (for the "two-type" case the memory is probably equal):

# cpdef if you want to use the enum from Python too
cdef enum FuncType:
    twoArg, threeArg

cdef union FuncPtrUnion:
    func2param f2p
    func3param f3p

cdef struct FuncPtrHolder:
    FuncType type_
    FuncPtrUnion value

Just to illustrate how you'd use it:

cdef long f(long x1, long x2):
    print("f",x1,x2)
    return 0

cdef long g(long x1, long x2, long x3):
    print("g",x1,x2,x3)
    return 1

def example():
    cdef FuncPtrHolder fArray[10]
    for i in range(10):
        if i%2:
            fArray[i].type_ = twoArg
            fArray[i].value.f2p = &f
        else:
            fArray[i].type_ = threeArg
            fArray[i].value.f3p = &g
    # use
    for i in range(10):
        if fArray[i].type_ == twoArg:
            fArray[i].value.f2p(i,i+1)
        elif fArray[i].type_ == threeArg:
            fArray[i].value.f3p(i,i+1,i+2)

It looks like in your code you have some list of Python objects and this is why you're getting compile errors. Without code it's impossible to know why, but I think Cython can automatically generate Python wrappers for cdef functions so I'd guess you've somehow made a list of those wrappers. If you want to handle Python lists of FuncPtrHolder (or otherwise use it from Python) you'd have to wrap it in a cdef class.

In this case my preferred, simpler solution would probably be to just use func3param and have the "two parameter" functions simply ignore the third argument. This means all the function pointers would have a consistent signature, which would make a lot more sense to me.

You have some interesting ideas here that I want to try. Having an unused parameter would work, of course, but I would prefer to not do it that way, because then adding more functions later requires me to always change all previous functions to use that max number of parameters. I'd also rather not use a struct for the same reason. — Bruce Nielson, Aug 21 '19 at 18:58
I'm unclear why you say "I can't think of anything useful you can do with an array of function pointers of unknown type" My question was specific to how to handle this with two known types. I'll store the number of parameters and then switch to use the right signature depending on that number. So I will know the type. I just can't figure out how to get it to cast to the needed type. I'll review your explanation on that above and try again. — Bruce Nielson, Aug 21 '19 at 18:59
@BruceNielson I think we probably agree on "array of function pointers of unknown type" since you say you're planning to store the number of parameters. I was just suggesting ways of storing it with the function points but you could of course store it in a different array. — DavidW, Aug 21 '19 at 19:14
Instead of "unused parameter" would an interface where you pass `(int num_args, long* array)` work instead to support any number of parameters? — DavidW, Aug 21 '19 at 19:16
I'd still like to know how to handle an array of functions, however, for the sake of my own learning. (And why casting didn't work.) — Bruce Nielson, Aug 21 '19 at 22:49

Cython: How to make an array of functions with different signatures

1 Answers1