Indexing static arrays in Cython

Question

I have a static array of doubles defined in a c++ file that I am trying to port over to Cython.

I followed the recommendation from Can I create a static C array with Cython? to use a hard-coded array of coefficients since I don't want to have to index a list if there is any way I can avoid it (very performance-sensitive segment of code so speed is quite important). Otherwise I could just use a list or something like that. Simple to code up, but too slow.

My array code implementation (in the .PXD file) looks like

cdef double *a_radial = [25932.1070099 , 0.914825434095 , -177.588568125 , -0.237052788124 , -172347.610527 , -12.0687599808 , -0.0128861161041 , -151.202604262 , -0.999674457769 , 0.0161435039267 , 0.825533456725]

and when I go to reference an element of the array (in the pure-python-mode .py file) like a_radial[0] I get a crash and no exception is produced. Is this a bug? Or am I doing something stupid?

Thanks, Ian

score 0 · Answer 1 · answered Dec 11 '12 at 22:35

It's somewhere in between. It's not supported and the Cython people know about it.

http://trac.cython.org/cython_trac/ticket/113

I tried two different things:

cdef get_list():
    cdef double *a_radial = [25932.1070099 , 0.914825434095 , -177.588568125 , -0.237052788124 , -172347.610527 , -12.0687599808 , -0.0128861161041 , -151.202604262 , -0.999674457769 , 0.0161435039267 , 0.825533456725]
    return a_radial

And

def get_list():
    cdef double *a_radial = [25932.1070099 , 0.914825434095 , -177.588568125 , -0.237052788124 , -172347.610527 , -12.0687599808 , -0.0128861161041 , -151.202604262 , -0.999674457769 , 0.0161435039267 , 0.825533456725]
    return a_radial

Both times it would not compile properly. It seems to be REALLY unsupported.

I was able to coerce Cython to allow me to make a static array, but only inside of a method call. That makes me think they know where to put the array in those cases (the stack) but at the module level, where does it go? The heap? What's the mechanism for allocating it? Also if you want to access the array in pure-python mode you'll need to convert it to something that python understands, and it doesn't understand a C array. You can only return Python objects or things which Cython can automatically turn into Python objects.

If performance is an issue then keeping this in the C-mode code shouldn't be a problem. If you need to get it to python-mode what's wrong with declaring a static array in one place and a python list as well. They can be a line apart to make updating easy.

cdef class Fun:
    a_radial = [25932.1070099 , 0.914825434095 , -177.588568125 , -0.237052788124 , -172347.610527 , -12.0687599808 , -0.0128861161041 , -151.202604262 , -0.999674457769 , 0.0161435039267 , 0.825533456725]
    def do_calcs(self):
        cdef double *a_radial = [25932.1070099 , 0.914825434095 , -177.588568125 , -0.237052788124 , -172347.610527 , -12.0687599808 , -0.0128861161041 , -151.202604262 , -0.999674457769 , 0.0161435039267 , 0.825533456725]
        print a_radial[0]

Can you not wrap all your performance sensitive stuff into a single python method call? I was able to also locate the static array inside a single function:

def do_calcs():
    cdef double *a_radial = [25932.1070099 , 0.914825434095 , -177.588568125 , -0.237052788124 , -172347.610527 , -12.0687599808 , -0.0128861161041 , -151.202604262 , -0.999674457769 , 0.0161435039267 , 0.825533456725]
    print a_radial[0]

Why does the static array need to be allocated outside of a function call?

I have resolved the problem through an unelegant solution of just hard coding the coefficients individually. Which works. I like the last code segment, but won't the array get allocated at each call to the function? That is what I had hoped to avoid by defining the array at the module-level. — ibell, Dec 12 '12 at 08:42
Yes it gets allocated every time, but it's allocated on the stack. So it's not a call to malloc, it's automagically handled for you. If there are only 5-10 coefficients it's nothing to worry about. If there are 1000, then yes it'll take a bit longer. — Mike Sandford, Dec 12 '12 at 14:59
I've profiled it and the allocation was actually taking a non-negligible amount of time. Reason being that the function is getting called millions of times, hence the millions of allocations of 5 element arrays which was a serious bottleneck. So it basically needs to be allocated at the module level so it gets allocated only once and then can be used without incurring any overhead at all — ibell, Dec 12 '12 at 20:08
Ahh, what about going the ctypes route then? It'll be statically allocated in the library (just once) and loaded in when you import it. It's not quite as nice as Cython in some ways, but if it's a single function call it's not bad. — Mike Sandford, Dec 13 '12 at 14:36
Sadly it is quite embedded within a cython architecture, so ctypes is out. At this point it is more an academic exercise than anything else. — ibell, Dec 13 '12 at 14:47

Indexing static arrays in Cython

1 Answers1