3

I am trying to do a time series prediction with ARIMA. So, as the first step, I am doing some series transformation

#Taking log transform
dflog=np.log(df)
#Taking exponential weighted mean`enter code here`
df_expwighted_mean = dflog.ewm(span=12).mean()
#Taking moving average
df_expwighted_mean_diff = dflog - df_expwighted_mean
#Differencing
df_diff = df_expwighted_mean_diff - df_expwighted_mean_diff.shift()
#filling zero for NaN
df_diff = df_diff.fillna(0)

And after with the below code I am very much able to reach back to the original series

# Take cumulative some to remove the differencing
bdf_expwighted_mean_diff = df_diff.cumsum()
# Add rolling mean as we originally reduced it
bdf_log=bdf_expwighted_mean_diff + df_expwighted_mean
#Take exponentiation as we originally did log transform
bdf=np.exp(bdf_log)

But the problem comes when I do this on the predicted series. It fails on it as I do not have the EWM of the predicted series.(pdf_expwighted_mean) SO basically, I want some way to reverse the exponentially weighted mean.

df_expwighted_mean = dflog.ewm(span=12).mean()

Any thoughts?

sashimi
  • 1,224
  • 2
  • 15
  • 23
Johnson Francis
  • 249
  • 3
  • 17

1 Answers1

3

It doesn't make sense to reverse exponentially weighted mean in Time series prediction. Exponentially weighted mean is used smoothen a time series, basically you are trying to remove noise from the series that would otherwise make the series hard to predict.

For Example: Let red series be your actual data, blue is the EWMA series, green is predicted series based on EWMA series in the following image ewm example

Once you use the smoothened series to predict, reversing EWMA would mean you add noise to it. You are able to it on source data becuase you stored the noise data from your original data. Usualy you just use the predictions on EWMA as is, ie. no reversing of EWMA required.

In your case, just do cumsum and exp(to reverse differencing and log).

Raghava Dhanya
  • 959
  • 15
  • 22
  • Thats exactly the problem I have. I want to go back to the original values after prediction with EWM. If I use just log transform or box cox transform or any mathematical single value transformation, I am able to go back. But this EWMA, I am unable to. Even WMA, I found some ways to do. – Johnson Francis Nov 24 '20 at 08:59
  • @JohnsonFrancis , I'm saying you can't and you shouldn't reverse transform. EWMA removes noise, Reversing it means adding noise. The prediction is still usable as is without EWMA reverse. You should think of EWMA as cleaning rather than a tranform. – Raghava Dhanya Nov 24 '20 at 10:53