numpy fast operation for slicing specific cells from a nd array

Asked May 27 '19 at 11:47

Active May 27 '19 at 12:59

Viewed 29 times

After profiling the following load data function, I've realized the following lines are a major bottleneck:

 dist_1 = dist[random_labels, :][:, random_labels]  
 dist_2 = dist[other_random_labels, :][:, other_random_labels]

where the size of dist is 6000,6000 and the random labels is of length 5000.
I'm trying to use np.take but

np.take(dist_1,[random_labels,random_labels]) == dist_1[random_labels, :][:, random_labels]

is False. where the dimention of np.take(dist_1,[random_labels,random_labels]) is (2,5000)
Is there an efficient way of doing this in numpy?
edit: this is the closest I've got:

 dist_1 = np.take(np.take(dist, random_labels, axis=0), random_labels, axis=1)

edited May 27 '19 at 12:03

asked May 27 '19 at 11:47

DsCpp

2,259
3
18
46

using np._ix produces "memory error" – DsCpp May 27 '19 at 12:03
Does - `np.ix_(random_labels,random_labels)` produce memory error or `dist[np.ix_(random_labels,random_labels)]`? – Divakar May 27 '19 at 12:13
np.ix_(random_labels,random_labels) – DsCpp May 27 '19 at 12:27
Try : `dist[random_labels[:,None],random_labels]`. – Divakar May 27 '19 at 12:29
Well, it's not faster than np.take(np.take(dist, random_labels, axis=0), random_labels, axis=1) – DsCpp May 27 '19 at 12:37
Yeah with selecting 5000 out of 6000 isn't helping with np.ix_. Reopened. – Divakar May 27 '19 at 12:59
Also, would `random_labels` be sorted? – Divakar May 27 '19 at 13:03
no, it's also a permutation – DsCpp May 27 '19 at 13:14

numpy fast operation for slicing specific cells from a nd array

0 Answers0