How to compute sum over a tuple with unknown length in Cython

Question

I want to understand why the following Cython code fails to compile. This function: dimsum simply computes sum of all dimensions of a numpy array:

from numpy cimport ndarray
import numpy as np

cpdef int dimsum(ndarray x):
    cdef:
        int N
        tuple shape
    
    shape = x.shape
    N = np.sum(shape)

    return N

The dimension of input array: x is unknown before run-time. As such, the shape of input array can not be unpacked and assigned to a few new variables.

I got the following error in the compilation:

Error compiling Cython file:
------------------------------------------------------------
...
cpdef int get_size(ndarray x):
    cdef:
        int N
        tuple shape

    shape = x.shape
            ^
------------------------------------------------------------

test_shape_tuple.pyx:9:13: Cannot convert 'npy_intp *' to Python object

Any suggestions on why this happens and how to resolve this?

Does this answer your question? [Numpy->Cython conversion: Compile error:Cannot convert 'npy\_intp \*' to Python object](https://stackoverflow.com/questions/9452427/numpy-cython-conversion-compile-errorcannot-convert-npy-intp-to-python-ob) — Dominik Stańczak, Aug 05 '22 at 03:49
The above does not answer my question. Instead, this link here (https://stackoverflow.com/questions/51737512/cython-why-do-numpy-arrays-need-to-be-type-cast-to-object) addressed this issue. It seems that the numpy array needs to be cast to object before calling the shape property. — Wei Li, Aug 05 '22 at 05:42
`shape = x.shape[:x.ndim]` might work (but I can't easily test it right now) — DavidW, Aug 05 '22 at 07:53

score 0 · Accepted Answer · answered Aug 05 '22 at 05:38

Despite your mysterious intent with this function, you can solve this error by assigning the npy_intp* type to the shape variable directly.

A secondary performance-related issue with your code is the use of the np.sum function that forces a conversion to Python object.

A more efficient code would be (note that you could avoid defining the shape variable and reading x.shape directly in the for loop):

import numpy as np
cimport numpy as np
from numpy cimport ndarray, npy_intp


cpdef int dimsum(ndarray x):
    cdef:
        int i, out
        npy_intp* shape
    with nogil:
        shape = x.shape
        out = 0
        for i in range(x.ndim):
            out += shape[i]
    return out

How to compute sum over a tuple with unknown length in Cython

1 Answers1