3

Say I have an array

data = np.arange(6)

I want to find the sum of the entire array and the second half using np.add.reduceat.1

If I do it like this:

np.add.reduceat(data, [0, 6, 3])[::2]

I immediately get an error

IndexError: index 6 out-of-bounds in add.reduceat [0, 6)

If I do it like this

np.add.reduceat(data, [0, 5, 3])[::2]

I get the wrong answer (10 should be 15):

array([10, 12])

The only solution I have been able to come up with is to mask the locations where the last index is necessary, subtract 1 from them, and then add the last element back in there:

index = np.array([0, 6, 3])
mask = (index == data.size)
index[mask] -= 1
result = np.add.reduceat(data, index)
# Mask is shifted back by one because it's the previous element that needs to be updated
result[:-1][mask[1:]] += data[-1]

Then result[::2] gives the desired answer. This looks like a giant kludge for something that I would expect to be an elegant one-liner (and faster than this).

1 I am fully aware that there are better ways to do this. This is just a contrived example for purposes of illustration. The real problem for this question originated with an attempt to solve numpy: fast regularly-spaced average for large numbers of line segments / points.

Mad Physicist
  • 107,652
  • 25
  • 181
  • 264

1 Answers1

4

I haven't used reduceat much, but it looks like you can only have one open ended range, one add to the end.

One way around that is to pad the array (yes, I do normally rail against using np.append :) ):

In [165]: np.add.reduceat(np.append(x,0),[0,6,3])
Out[165]: array([15,  0, 12])

or with a full pairing of ranges:

In [166]: np.add.reduceat(np.append(x,0),[0,6,3,6])
Out[166]: array([15,  0, 12,  0])

I omitted the usual [::2] to clarify what is going on.

hpaulj
  • 221,503
  • 14
  • 230
  • 353
  • I know that it's traditional to wait before accepting, but this is pretty much exactly what I was looking for. Especially since the negative increment is clearly defined in this case. – Mad Physicist Sep 13 '18 at 18:57
  • In case you were wondering what I ended up using this for: https://stackoverflow.com/a/52320926/2988730 – Mad Physicist Sep 13 '18 at 19:59
  • 3
    I was writing a github issue when I found this answer. Why isn't this considered a bug? Appending additional elements works, but I see no reason to exclude [`indices[i] >= len(array)`](https://numpy.org/doc/stable/reference/generated/numpy.ufunc.reduceat.html).The `=` looks to me like a *careless mistake* for slices. – Michael Szczesny Apr 09 '22 at 00:40
  • I think there are open issues in GitHub regarding this https://github.com/numpy/numpy/issues/834 and https://github.com/numpy/numpy/issues/835 – Jaakko Aug 05 '22 at 05:57