3

I've scoured stackoverflow, and I can't find exactly what fits the bill for me.

I'm calculating a weighted moving average for a rolling window.

The equation is: 

 #weighted average temp with smoothing factor, a
 #T_w = sum[k=1,24](a^(k-1)*T(t-k)) / sum[k=1,24]a^(k-1)

Seems easy enough, but I need to apply this average to a rolling window. I can do rolling mean (simple moving average):

 T_ = pd.DataFrame()
 T_ = temps['T'].rolling(window=24).mean()

But now I want to apply weights for ONLY the window I'm averaging over. Python's .ewm() doesn't cut the mustard, because I want the weights to be just for the window I'm "rolling" over.

I've found a few snippets that seem like they might work, but components fail:

from functools import partial

window = 13
alpha = 1-ln(2)/3    # This is ewma's decay factor.
weights = list(reversed([(1-alpha)**n for n in range(window)]))
ewma = partial(average, weights=weight)
rolling_average = series.rolling(window).apply(ewma)

Here, the problem I've run into is how partial() calls average() -- this was introduced here - Create a rolling custom EWMA on a pandas dataframe - but I can't comment yet (newb), and I don't know where to take this.

Another solution I have implemented, but it doesn't do exactly what I need:

alpha = 0.1    # This is my smoothing parameter
weights = list(reversed([(1-alpha)**n for n in range(window)]))
def f(w):
    def g(x):
        return (w*x).mean()
    return g
T_ = pd.DataFrame()
T_ = temps['T'].rolling(window=24).apply(f(weights))

Based on a proposed solution here: Calculating weighted moving average using pandas Rolling method The problem with this approach is that it calculates the mean, whereas I need effectively something like this:

return (w*x).sum() / w.sum()

But that doesn't work, because

AttributeError: 'list' object has no attribute 'sum'

How do I calculate a rolling weighted moving average with a specified window (here, the last 24 hours), and a specified smoothing parameter a (which is only applied to the last 24 hours) ?

bearcub
  • 313
  • 4
  • 11
  • 1
    Have you considered [`np.average()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.average.html)'s weights attribute? – gosuto Sep 09 '18 at 07:19

2 Answers2

5

Instead of return (w*x).sum() / w.sum(), try sum(w*x) / sum(w). This will get you past the AttributeError at least.

sum is a Python built-in that will work on any iterable holding objects that can be summed (i.e. added objects that can be added together with +). Examples of these objects include int, float, etc.

Matt Messersmith
  • 12,939
  • 6
  • 51
  • 52
1

you can set the raw=True in .apply function. it makes the input as ndarray. then you can multiply that with normalize weights: temps.rolling(24, axis=1).apply(lambda x: np.sum(x * w), raw=True)

Mahsa Alizadeh
  • 11
  • 1
  • Per the [pandas docs](https://pandas.pydata.org/docs/reference/api/pandas.core.window.rolling.Rolling.apply.html) "the passed function will receive ndarray objects instead. If you are just applying a NumPy reduction function this will achieve much better performance." – Raisin Jan 13 '23 at 21:57