Why NumPy creates a view for x[[slice(None), 1, 2]]

Question

In NumPy documentation for advanced indexing, it is mentioned that

Also recognize that x[[1, 2, 3]] will trigger advanced indexing, whereas x[[1, 2, slice(None)]] will trigger basic slicing.

A matrix is stored sequentially into the memory. I understand that it makes sense to make a view of x[[1, 2, slice(None)]] since the elements are stored sequentially into the memory. But why Numpy returns a view of x[[1, slice(None), 2]] or x[[slice(None), 1, 2]]. For instance, assume

x = [[[ 0,  1,  2],
      [ 3,  4,  5],
      [ 6,  7,  8]],
     [[ 9, 10, 11],
      [12, 13, 14],
      [15, 16, 17]],
     [[18, 19, 20],
      [21, 22, 23],
      [24, 25, 26]]]

x[[1, slice(None), 2]] returns a view of [11, 14, 17] which is not sequentially stored in the memory as well as for x[[slice(None), 1, 2]] which returns [5, 14, 23].

I would like to know

1. Why NumPy even returns a view in these two cases

2. How NumPy handles memory addressing to create these views

Answer 1

From the SciPy cookbook:

The rule of thumb for creating a slice view is that the viewed elements can be addressed with offsets, strides, and counts in the original array.

When you have an indexing like x[[1, slice(None), 2]], you get a view because slicing an entire axis allows for a certain offset, stride and count to represent the slice with the original array.

For instance, with x = np.arange(27).reshape(3, 3, 3).copy(), we have:

In [79]: x_view = x[1, :, 2]  # or equivalently x[[1, slice(None), 2]]

In [80]: x_view
Out[80]: array([11, 14, 17])

In [81]: x_view.base
Out[81]: 
array([[[ 0,  1,  2],
        [ 3,  4,  5],
        [ 6,  7,  8]],

       [[ 9, 10, 11],
        [12, 13, 14],
        [15, 16, 17]],

       [[18, 19, 20],
        [21, 22, 23],
        [24, 25, 26]]])

Then we can use numpy.byte_bounds (not part of the public API, YMMV) to illustrate the offset to get our slice from our original array.

In [82]: np.byte_bounds(x_view)[0] - np.byte_bounds(x_view.base)[0]
Out[82]: 88

This makes sense, since there are 11 8-byte integers before the first value in the slice, 11. NumPy calculates this offset with a formula you can see here, using the strides of the original array.

In [93]: (x.strides * np.array([1, 0, 2])).sum()
Out[93]: 88

The strides in our slice simply become whatever the strides were for x along the axis (or axes) on which we are slicing. i.e. x.strides[1] == x_view.strides[0]. Now together the offset, new strides and count are enough information for NumPy to view our slice from our original array.

In [94]: x_view.strides
Out[94]: (24,)

In [95]: x_view.size
Out[95]: 3

Finally, the reason why you trigger fancy indexing with x[[0, 1, 2]] for instance is because in the absence of a full axis slice, it isn't generally possible to formulate some new offset, byte order, strides and count such that we can view the slice with the same underlying data.

Answer 2

I like to use __array_interface__ to examine the attributes of an array:

With your x:

In [51]: x.__array_interface__
Out[51]: 
{'data': (43241792, False),
 'strides': None,
 'descr': [('', '<i8')],
 'typestr': '<i8',
 'shape': (3, 3, 3),
 'version': 3}
In [52]: x.strides
Out[52]: (72, 24, 8)

It's a (3,3,3) array. The last axis can be scanned by stepping 8 bytes at a time, the size of x.itemsize. 3*8 steps rows, and 3*3*8 steps through the planes (1st dim).

In [53]: y = x[:,1,2]
In [54]: y.shape
Out[54]: (3,)
In [55]: y.strides
Out[55]: (72,)
In [56]: y.__array_interface__['data']
Out[56]: (43241832, False)

y elements can addressed by stepping by planes, 3*3*8. 43241832 is the starting point, 40 bytes into the data buffer, 5*8

In [59]: y
Out[59]: array([ 5, 14, 23])

So it starts at the 5th element, and steps forward one plane at a time (9 elements), for a total of 3 elements.

The fact that y.__array_interface__['data'] falls within the range of the x 'data' tells me that y is a view. It's a view because a combination of this buffer starting point, the strides and the shape let us access all the values of y.

With advanced indexing it isn't possible to (in general) to access the elements with these simple parameters, so numpy has to make a copy of the data.

---

A reversed view is possible by just changing the strides and 'data' start point:

In [60]: z = y[::-1]
In [61]: z.__array_interface__
Out[61]: 
{'data': (43241976, False),
 'strides': (-72,),
 'descr': [('', '<i8')],
 'typestr': '<i8',
 'shape': (3,),
 'version': 3}

Transpose also changes strides:

In [62]: x.T.strides
Out[62]: (8, 24, 72)