2

Can someone please help me to understand why sometimes the advanced selection doesn't work and what I can do to get it to work (2nd case)?

>>> import numpy as np
>>> b = np.random.rand(5, 14, 3, 2)

# advanced selection works as expected
>>> b[[0,1],[0,1]]
array([[[ 0.7575555 ,  0.18989068],
        [ 0.06816789,  0.95760398],
        [ 0.88358107,  0.19558106]],

       [[ 0.62122898,  0.95066355],
        [ 0.62947885,  0.00297711],
        [ 0.70292323,  0.2109297 ]]])

# doesn't work - why?
>>> b[[0,1],[0,1,2]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: shape mismatch: objects cannot be broadcast to a single shape

# but this seems to work
>>> b[:,[0,1,2]]
array([[[[  7.57555496e-01,   1.89890676e-01],
         [  6.81678915e-02,   9.57603975e-01],
         [  8.83581071e-01,   1.95581063e-01]],

        [[  2.24896112e-01,   4.77818599e-01],
         [  4.29313861e-02,   8.61578045e-02],
         [  4.80092364e-01,   3.66821618e-01]],
...

Update

Breaking up the selection seems to resolve the problem, but I am unsure why this is necessary (or if there's a better way to achieve this).

>>> b.shape
(5, 14, 3, 2)
>>> b[[0,1]].shape
(2, 14, 3, 2)

# trying to separate indexing by dimension.
>>> b[[0,1]][:,[0,1,2]]
array([[[[ 0.7575555 ,  0.18989068],
         [ 0.06816789,  0.95760398],
         [ 0.88358107,  0.19558106]],

        [[ 0.22489611,  0.4778186 ],
         [ 0.04293139,  0.0861578 ],
orange
  • 7,755
  • 14
  • 75
  • 139
  • 1
    What do you think `b[[0,1],[0,1,2]]` is supposed to do? If you expected that to work, then `b[[0,1],[0,1]]` most likely isn't doing what you wanted either, and you just didn't notice due to the lack of a noisy error message. – user2357112 Apr 20 '16 at 04:01
  • 1
    Well, for starters it should not result in an error at all. – orange Apr 20 '16 at 04:03
  • 1
    This index should select elements `0` and `1` from dimension `0`, and `0`, `1` and `2` from dimension `1` (and `slice(None)` for the other subsequent dimensions). – orange Apr 20 '16 at 04:04

2 Answers2

5

You want

b[np.ix_([0, 1], [0, 1, 2])]

You also need to do the same thing for b[[0, 1], [0, 1]], because that's not actually doing what you think it is:

b[np.ix_([0, 1], [0, 1])]

The problem here is that advanced indexing does something completely different from what you think it does. You've made the mistake of thinking that b[[0, 1], [0, 1, 2]] means "take all parts b[i, j] of b where i is 0 or 1 and j is 0, 1, or 2". This is a reasonable mistake to make, considering that it seems to work that way when you have one list in the indexing expression, like

b[:, [1, 3, 5], 2]

In fact, for an array A and one-dimensional integer arrays I and J, A[I, J] is an array where

A[I, J][n] == A[I[n], J[n]]

This generalizes in the natural way to more index arrays, so for example

A[I, J, K][n] == A[I[n], J[n], K[n]]

and to higher-dimensional index arrays, so if I and J are two-dimensional, then

A[I, J][m, n] == A[I[m, n], J[m, n]]

It also applies the broadcasting rules to the index arrays, and converts lists in the indexes to arrays. This is much more powerful than what you expected to happen, but it means that to do what you were trying to do, you need something like

b[[[0],
   [1]], [[0, 1, 2]]]

np.ix_ is a helper that will do that for you so you don't have to write a dozen brackets.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • Thanks for this comprehensive and insightful explanation! – orange Apr 20 '16 at 04:21
  • But how can I use np.ix_ when there are ':' in my selection. Example a[:, [0,1], [0, 1, 2]] when I try a[:, np.ix_([0, 1], [0, 1, 2])] it just complains. –  Feb 01 '17 at 15:53
  • @Nozdrum: Use `np.arange(a.shape[0])` instead of `:` (with `np.ix_`), or use `:2` and `:3` instead of `[0, 1]` and `[0, 1, 2]` (without `np.ix_`). Mixing basic and advanced indexing is weird and tricky; in all but the most simple cases, it's best to keep basic and advanced indexing separate. – user2357112 Feb 01 '17 at 16:59
  • @user2357112 Thanks for the answer, the problem I have that it just becomes unreadable. Like really unreadable. I have tons of lines that look like this: delay[:, :, indices, range_i[:5]] = channels.delay[:, :, indices, range_j[5:10]; For once in my life i think matlab is doing something better than numpy. –  Feb 01 '17 at 17:12
  • 1
    @Nozdrum: It could help to rearrange the axes of `delay` and `channels.delay` to put the axes you need to apply advanced indexing to at the front. Then you can just omit the `:`s. (You can make an axes-rearranged view with `np.transpose` and keep `delay` and `channels.delay` with their original shapes.) After looking up how Matlab handles this, I can definitely agree that Matlab makes your desired operation easier. – user2357112 Feb 01 '17 at 17:25
  • @user2357112 Thanks! Transposing here helps a lot in making it readable and easy to write. Also tranpose doesnt seem to create a copy of the data, so that I can just use whatever form I need at the moment. –  Feb 01 '17 at 18:05
0

I think you misunderstood the advanced selection syntax for this case. I used your example, just made it smaller to be easier to see.

import numpy as np
b = np.random.rand(5, 4, 3, 2)

# advanced selection works as expected
print b[[0,1],[0,1]]   # http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html
                       # this picks the two i,j=0 (a 3x2 matrix) and i=1,j=1, another 3x2 matrix

# doesn't work - why?
#print  b[[0,1],[0,1,2]]   # this doesnt' work because [0,1] and [0,1,2] have different lengths

print b[[0,1,2],[0,1,2]]  # works

Output:

[[[ 0.27334558  0.90065184]
  [ 0.8624593   0.34324983]
  [ 0.19574819  0.2825373 ]]

 [[ 0.38660087  0.63941692]
  [ 0.81522421  0.16661912]
  [ 0.81518479  0.78655536]]]
[[[ 0.27334558  0.90065184]
  [ 0.8624593   0.34324983]
  [ 0.19574819  0.2825373 ]]

 [[ 0.38660087  0.63941692]
  [ 0.81522421  0.16661912]
  [ 0.81518479  0.78655536]]

 [[ 0.65336551  0.1435357 ]
  [ 0.91380873  0.45225145]
  [ 0.57255923  0.7645396 ]]]
roadrunner66
  • 7,772
  • 4
  • 32
  • 38