Cumulative sum of variable till a given percentile

Question

I would like to sum up all the values in an array till a given percentile. E.g.

import numpy as np
a = [15, 40, 124, 282, 914, 308]
print np.percentile(a,90)

The 90th percentile is ~611 and the cumulative sum till then is 461

Is there any function in Python which can do that?

score 5 · Answer 1 · answered Apr 01 '14 at 01:24

5

import numpy as np
a = np.array([15, 40, 124, 282, 914, 308])
b = np.cumsum(a)
p90 = np.percentile(a, 90)
print b[b < p90][-1] #461

answered Apr 01 '14 at 01:24

JoshAdel

66,734
27
141
140

1

Since the cumsum array is sorted by definition you could searchsort the percentile value. – Jaime Apr 01 '14 at 01:59
Thanks! Esp the helpful comments re which solution might be faster in which circumstance... – user308827 Apr 01 '14 at 03:36

score 4 · Answer 2 · answered Apr 01 '14 at 01:13

4

None that I know of, but you can do this

import numpy as np
from itertools import takewhile

a = [15, 40, 124, 282, 914, 308]
p90 = np.percentile(a,90)
print sum(takewhile(lambda x : x < p90,  a))

Out:

answered Apr 01 '14 at 01:13

JaminSore

3,758
1
25
21

score 4 · Accepted Answer · answered Apr 01 '14 at 01:25

4

A=np.array(a)
A[:(A<np.percentile(a, 90)).argmin()].sum() #461

@JoshAdel's

%%timeit
    ...: b = np.cumsum(a)
    ...: p90 = np.percentile(a, 90)
    ...: b[b < p90][-1]
    ...: 
1000 loops, best of 3: 217 µs per loop

This:

%timeit A[:(A<np.percentile(a, 90)).argmin()].sum()
10000 loops, best of 3: 191 µs per loop

answered Apr 01 '14 at 01:25

CT Zhu

52,648
17
120
133

+1 nice solution. I will say that on my machine, for the given system size, my code is a bit faster. However for larger arrays, depending on the composition, your code can be slightly (5-10%) faster. – JoshAdel Apr 01 '14 at 01:50
I'll also add that on my machine, the itertools solution is marginally faster for a large random array than either numpy solution. It's a bit slower for small arrays. Again this is just on my machine and timings are bound to vary a little. I'll leave it the OP to decide what is best for their particular hardware/array composition. – JoshAdel Apr 01 '14 at 01:56
Very interesting Josh, I just found `itertools` to be always marginally faster than either one of ours. I have `MKL` on `win32` platform, which I am not sure if these operations even make use of. A bit of information for whoever understand what's going on under the hood. – CT Zhu Apr 01 '14 at 02:47

Cumulative sum of variable till a given percentile

3 Answers3