1

I currently have a dataframe formatted as such:

 a b c e f
 2 3 4 5 1
 9 8 6 4 6
 1 2 2 2 3
 9 8 8 9 1
 3 6 8 6 2
 . . . . . 
 . . . . . 
 7 5 4 1 8

My goal is to apply an adaptive weight function over the last n(4) games in a sliding window. On each individual column. For example:

awa = [0.1, 0.2, 0.3, 0.4])
col a 
a5 = sum([2, 9, 1, 9] * awa)

So the 5th element would be equal to the sum of last 4 (.2 + 1.8 + .3 + 3.6) = 5.9 I want to do this while maintaining the original dataset - meaning the 5.9 calculation wouldn't be used in the AWA calculation for the next elements. Next calculation:

a6 = sum([9, 1, 9, 3] * awa)
a6 = 5

And so forth...

My initial approach would be the following:

  1. Convert the dataframe to a numpy array
df.to_numpy()
  1. Utilize the np.convolve function OR sliding window view
from numpy.lib.stride_tricks import sliding_window_view
np.sum(sliding_window_view(values, window_shape = 4), axis = 1)
np.convolve(mydata,np.ones(4,dtype=int),'valid')

My problem becomes twofold:

  1. How do utilize these functions on columns instead of numpy rows?
  2. How to apply the adaptive weight function dynamically to each 'column slice' before remaking the frame?
Sentient AI Turing
  • 49
  • 4
  • 3
    Would `.rolling` help at all? https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.rolling.html Perhaps: `df.rolling(4).apply(lambda x: x @ awa)` – Chrysophylaxs Mar 28 '23 at 20:21
  • @Chrysophylaxs I was about to suggest it, you should post it as answer ;) – mozway Mar 28 '23 at 20:24
  • @Chrysophylaxs Where would rolling store the values? Would it incorporate the adaptive weight average(s) in the next calculation? – Sentient AI Turing Mar 28 '23 at 20:29
  • 1
    As far as I know, rolling does not store any values, it returns you a Rolling object which, when prompted for some calculation (`.sum`, `.mean`, `.apply` etc.), computes the boundaries of each window on the fly, and passes the subframes to the aggregator. You can for instance create your own Indexer (https://pandas.pydata.org/pandas-docs/stable/user_guide/window.html#custom-window-rolling) that computes the rolling window boundaries. That said, I am not an expert on pandas internals, so you'd have to dig in the code to know for sure. – Chrysophylaxs Mar 28 '23 at 21:54

1 Answers1

1

Try using rolling. For example something along these lines:

df['a_r'] = df['a'].rolling(4).agg(lambda x: x@awa)
Guru Stron
  • 102,774
  • 10
  • 95
  • 132