Say I have a slice like x[p:-q:n] or x[::n] I want to use this to generate the index to be passed into numpy.ufunc.reduceat(x, [p, p + n, p + 2 * n, ...]) or numpy.ufunc.reduceat(x, [0, n, 2 * n, ...]). What is the easiest and efficient way to get it done?
Asked
Active
Viewed 302 times
0
Suminda Sirinath S. Dharmasena
- 1,831
- 3
- 26
- 35
-
Why not just use `range`? `list(range(0, len(x), n))` – Psidom Jan 11 '17 at 16:28
-
Is this the most efficient? – Suminda Sirinath S. Dharmasena Jan 11 '17 at 16:30
-
It's efficient if you want a list of indices. – Psidom Jan 11 '17 at 16:35
-
This runs inside a very large loop. One thing that worries me is creating a `range` and `list` object. But thanks and will use this if I do not get a more optimised way. – Suminda Sirinath S. Dharmasena Jan 11 '17 at 16:41
-
Try `np.arange` or `np.r_` – hpaulj Jan 11 '17 at 16:42
-
If you are working with regular intervals, simply slice, reshape and sum. – Divakar Jan 11 '17 at 17:21
1 Answers
3
Building on the comments:
In [351]: x=np.arange(100)
In [352]: np.r_[0:100:10]
Out[352]: array([ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90])
In [353]: np.add.reduceat(x,np.r_[0:100:10])
Out[353]: array([ 45, 145, 245, 345, 445, 545, 645, 745, 845, 945], dtype=int32)
In [354]: np.add.reduceat(x,np.arange(0,100,10))
Out[354]: array([ 45, 145, 245, 345, 445, 545, 645, 745, 845, 945], dtype=int32)
In [355]: np.add.reduceat(x,list(range(0,100,10)))
Out[355]: array([ 45, 145, 245, 345, 445, 545, 645, 745, 845, 945], dtype=int32)
In [356]: x.reshape(-1,10).sum(axis=1)
Out[356]: array([ 45, 145, 245, 345, 445, 545, 645, 745, 845, 945])
and timing:
In [357]: timeit np.add.reduceat(x,np.r_[0:100:10])
The slowest run took 9.30 times longer than the fastest. This could mean that an intermediate result is being cached.
10000 loops, best of 3: 31.2 µs per loop
In [358]: timeit np.add.reduceat(x,np.arange(0,100,10))
The slowest run took 85.75 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 6.69 µs per loop
In [359]: timeit np.add.reduceat(x,list(range(0,100,10)))
The slowest run took 4.31 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 11.9 µs per loop
In [360]: timeit x.reshape(-1,10).sum(axis=1)
The slowest run took 5.57 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 11.5 µs per loop
reduceat with arange looks best, but it should be tested on more realistic data. Speeds aren't that different at this size.
The value of r_ is that it lets you use the convenient slicing notation; it's in a file called index_tricks.py.
With a 10000 element x, times are 80, 46, 238, 51.
hpaulj
- 221,503
- 14
- 230
- 353