creating a 3d numpy array from a non-contigous set of contigous 2d slices

Question

Is it possible to use PyArray_NewFromDescr to create numpy array object from a set of contiguous 2d arrays, without copying the data?

Answer 1

There's a slight inconsistency in your post; I'm assuming your title describes what you want. Looking at the Python API docs, PyArray_NewFromDescr accepts just a single pointer to preallocated memory, so the answer is almost certainly no. It seems that you might be able to use the strides argument in a clever way to achieve this; but even so, the non-contiguous memory would have to be laid out in a regular way.

Furthermore, that's clearly not what numpy wants you to do, judging from PyArray_CheckStrides, which appears to assume a single block of contiguous memory.

Answer 2

Short answer, no.

Numpy expects all the data to be laid it in a simple, strided pattern. When iterating over the array, to advance in a dimension, it adds a constant, the stride size for that dimension, to the position in memory. So unless your 2-d slices are laid out regularly (e.g. every other row of a larger 3-d array), numpy will need to copy the data.

If you do have that order, you can do what you'll want. You'll need to make a PyArray struct where the data points to the first item, the strides are correct for layout, and the descr is correct as well. Most importantly, you'll want to set the base member to another python object to keep your big chunk of memory alive while this view exists.