2

I am observing that if a is a list (or a numpy array) with elements [1,2,3] and I ask for a[1:-1:-1], then I get the empty list. I would expect to get [2,1] assuming that the slicing spans the indexes obtainable decrementing from 1 to -1 excluding the last value (that is excluding -1), that is indexes 1 and 0.

The actual behavior may have some justification but makes things more complex than expected when one needs to take a subarray of an array a starting from some generic index i to index i+m (excluded) in reverse order. One would tend to write a[i+m-1:i-1:-1] but this suddenly breaks if i is set to 0. The fact that it works for all i but zero looks like a nasty inconsistency. Obviously, there are workarounds:

  1. one could write a[i+m-1-n:i-1-n:-1] offsetting everything by -n where n is the array length; or
  2. one could write a[i:i+m][::-1].

However, in case 1 the need to know the array length appears rather unnatural and in case 2 the double indexing appears as a not very justified overhead if the slicing is done in a tight loop.

  • Is there any important reason that I am missing for which it is important that the behavior is as it is?

  • Has this issue been considered by the NumPy community?

  • Is there some better workaround than those I came up with?

callegar
  • 223
  • 1
  • 2
  • 7
  • For Numpy arrays `a[i:i+m][::-1]` generates a view of the underlying array, i.e. it has negligible overhead and thus appears to be a valid solution. – a_guest Apr 23 '22 at 14:42

4 Answers4

3

-1 as an index has a special meaning [1], it's replaced with the highest possible = last index of a list. So a[1:-1:-1] becomes a[1:2:-1] which is empty.

[1] Actually, all negative indices in Python work like that. -1 means the last element of a list, -2 the second-to-last, -3 the one before that and so on.

Banana
  • 2,295
  • 1
  • 8
  • 25
  • 1
    While `::-1` effectively reverses a sequence, it is not true that `-1` as a step reverses the order of start and stop indices. `a[1:2:-1]` is never converted to `a[2:1]`. By this logic `a[2:0:-1]` would evaluate to `[1, 2]`. – a_guest Apr 23 '22 at 15:43
3

Numpy has adopted this behavior from Python's sequence indexing for which the rules are explained here (for some history see below). Specifically footnote (5) reads:

The slice of s from i to j with step k is defined as the sequence of items with index x = i + n*k such that 0 <= n < (j-i)/k. In other words, the indices are i, i+k, i+2*k, i+3*k and so on, stopping when j is reached (but never including j). When k is positive, i and j are reduced to len(s) if they are greater. When k is negative, i and j are reduced to len(s) - 1 if they are greater. If i or j are omitted or None, they become “end” values (which end depends on the sign of k). Note, k cannot be zero. If k is None, it is treated like 1.

So the indices are generated from multipliers n subject to 0 <= n < (j-i)/k. For your specific example (j-i)/k < 0 and hence no indices are computed.

For Numpy arrays a[i:i+m][::-1] generates a view of the underlying array, i.e. it has negligible overhead and thus appears to be a valid solution. It clearly conveys the intent, namely "take a subarray of an array a starting from some generic index i to index i+m (excluded) in reverse order".

Alternatively, you can use None as the stop argument if i is zero:

a[i+m-1:(None if i==0 else i-1):-1]

History

Originally, Python implemented slicing syntax via __getslice__ (see also here) which didn't allow a step argument, i.e. it only used the 2-argument form: a[i:j]. This was implemented by built-in sequences such as list. Back then, around 1995, the predecessor of Numpy, Numerical Python, was developed and discussed within the MATRIX-SIG (special interest group). This predecessor implemented a specific Slice type which could be used to also specify a so called stride (now step) in a form very similar to today's slice: e.g. a[Slice(None, None, 2)]. It was asked to extend Python's syntax to allow for the 3-form slicing known today: a[::2] (see e.g. this thread). This got implemented in form of the slice type and would be passed to __getitem__ instead of __getslice__. So back then, a[i:j] was resolved as a.__getslice__(i, j) while a[i:j:k] was resolved as a.__getitem__(slice(i, j, k)). Back then, Numerical Python even allowed "reverse" slicing with the 2-form, interpreting the second argument as the stride (see the docs; e.g. a[i:-1] was equivalent to a[i::-1] for an array object a). Indexing of arrays was oriented at how indexing for Python sequences worked: including the start index, excluding the stop index (see here). This applied to negative stride (step) as well, hence providing the behavior that can be observed today. The decision was probably based on the principle of least surprise (for "standard" Python users). It took a long time until Python 2.3 where the extended slicing feature including a step was implemented for the built-in types (see what's new and the docs; note that the 2.3 version of the docs contained a wrong description of slicing with step which was fixed for the 2.4 release).

a_guest
  • 34,165
  • 12
  • 64
  • 118
  • The behavior is clear and I like your solution with the nested if... Offsetting the index by minus the array length plus one is shorter but possibly less readable. Still I cannot fully understand the rationale for the original choice made in Python for indexes, since it makes some applications (notably signal processing ones) more prone to subtle issues. Having the indexes always interpreted modulo the array length when slicing would have probably been easier to understand and deploy. Is the current choice more efficient? – callegar Apr 23 '22 at 17:18
  • @callegar I edited my answer, adding some history. The extended slice notation with `step` was introduced long ago and was based on how the predecessor of Numpy, Numerical Python, handled indexing with stride (now called step). They used `Slice(start, stop, step)` and then asked to extend Python slicing syntax from `start:stop` to `start:stop:step` via `slice(start, stop, step)`. Since a `slice` object indicates where *indexing* starts and stops (even today, the docs of `slice` refer to *"the set of indices specified by `range(start, stop, step)`"*), this is always true, also for `step < 0`. – a_guest Apr 23 '22 at 20:19
1

List[1:-1:-1] means List[start index : end index : jump]

Indexing in List:

Number 1 2 3
Index 0 1 2
Index -3 -2 -1

So, if we take list a[1,2,3] and find list of a[1:-1:-1] means starting index = 1, ending index = -1, jump = -1

So, list traversing through the

index 1 (i.e. number=2) to index -1 (i.e. number=3) but jump = -1 (means backward position)

So, return an empty list i.e. []

AkashSingh3031
  • 36
  • 2
1

As others noted -1 as end point has special meaning

In [66]: a=[1,2,3]

Slice back to the beginning is best done with None:

In [68]: a[1::-1]
Out[68]: [2, 1]
In [69]: a[1:None:-1]
Out[69]: [2, 1]

Working with slices that could cross boundaries, either side can be tricky:

In [75]: [a[i+2-1:i-1:-1] for i in range(4)]
Out[75]: [[], [3, 2], [3], []]

simplify a bit:

In [77]: [a[i+2:i:-1] for i in range(-1,3)]
Out[77]: [[], [3, 2], [3], []]

We can correct the lower boundary by using a if clause:

In [78]: [a[i+2:None if i<0 else i:-1] for i in range(-1,3)]
Out[78]: [[2, 1], [3, 2], [3], []]
hpaulj
  • 221,503
  • 14
  • 230
  • 353