Best Way to Append C Arrays (or any arrays) in Cython

Question

I have a function in an inner loop that takes two arrays and combines them. To get a feel for what it's doing look at this example using lists:

a = [[1,2,3]]
b = [[4,5,6],
     [7,8,9]]

def combinearrays(a, b):
    a = a + b
    return a


def main():
    print(combinearrays(a,b))

The output would be:

[[1, 2, 3], [4, 5, 6], [7, 8, 9]]

The key thing here is that I always have the same number of columns, but I want to append rows together. Also, the values are always ints.

As an added wrinkle, I cheated and created a as a list within a list. But in reality, it might be a single dimensional array that I want to still combine into a 2D array.

I am currently doing this using Numpy in real life (i.e. not the toy problem above) and this works. But I really want to make this as fast as possible and it seems like c arrays should be faster. Obviously one problem with c arrays if I pass them as parameters, is I won't know the actual number of rows in the arrays passed. But I can always add additional parameters to pass that.

So it's not like I don't have a solution to this problem using Numpy, but I really want to know what the single fastest way to do this is in Cython. Since this is a call inside an inner loop, it's going to get called thousands of times. So every little savings is going to count big.

One obvious idea here would be to use malloc or something like that.

Answer 1

While I'm not convinced this is the only option, let me recommend the simple option of building a standard Python list using append and then using np.vstack or np.concatenate at the end to build a full Numpy array.

Numpy arrays store all the data essentially contiguously in memory (this isn't 100% true for if you're taking slices, but for freshly allocated memory it's basically true). When you resize the array it may get lucky and have unallocated memory after the array and then be able to reallocate in place. However, in general this won't happen and the entire contents of the array will need to be copied to the new location. (This will likely apply for any solution you devise yourself with malloc/realloc).

Python lists are good for two reasons:

1. They are internally a list of PyObject* (in this case to the Numpy arrays it contains). If copying is needed during the resize you are only copying the pointers to the arrays, and not the whole arrays.
2. They are designed to handle resizing/appending intelligently by over-allocating the space needed, so that they need only re-allocate more memory occasionally. Numpy arrays could have this feature, but it's less obviously a good thing for Numpy than it is for Python lists (if you have a 10GB data array that barely fits in memory do you really want it over-allocated?)

My proposed solution uses the flexibly, easily-resized list class to build your array, and then only finalizes to the inflexible but faster Numpy array at the end, therefore (largely) getting the best of both.

---

A completely untested outline of the same structure using C to allocate would look like:

 from libc.stdlib cimport malloc, free, realloc

 cdef int** ptr_array = NULL
 cdef int* current_row = NULL

 # just to be able to return a numpy array
 cdef int[:,::1] out

 rows_allocated = 0
 try:
     for row in range(num_rows):
         ptr_array = realloc(ptr_array, sizeof(int*)*(row+1))

         current_row = ptr_array[r] = malloc(sizeof(int)*row_length)
         rows_allocated = row+1
         # fill in data on current_row

     # pass to numpy so we can access in Python. There are other
     # way of transfering the data to Python...
     out = np.empty((rows_allocated,row_length),dtype=int)         
     for row in range(rows_allocated):
         for n in range(row_length):
             out[row,n] = ptr_array[row][n]

     return out.base
 finally:
     # clean up memory we have allocated
     for row in range(rows_allocated):
         free(ptr_array[row])
     free(ptr_array)

This is unoptimized - a better version would over-allocate ptr_array to avoid resizing each time. Because of this I don't actually expect it to be quick, but it's meant as an indication of how to start.