3

This time my question is more methodological than technical. I have weekly time series data which gets updated every week. Unfortunately the time series is quite volatile. I would thus like to apply a filter/a smoothing method. I tried Hodrick-Prescott and LOESS. Both results look fine, with the downturn that if a new datapoint follows which diverges strongly from the historic data points, the older values have to be revised/are changing. Does somebody know a method which is implemented in R, which could do what I want? A name of a method/a function would probably be completely sufficient. It should however be something more sophisticated than a left sided moving average, because I would not like to lose data at the beginning of the time series. Every helping comment is appreciated! Thank you very much!

Best regards,

Andreas

chameau13
  • 626
  • 7
  • 24

2 Answers2

4

I think (?) that the term you may be looking for is causal filtering, i.e. filtering that doesn't depend on future values. Within this category probably the simplest/best known approach is exponential smoothing, which is implemented in the forecast and expsmooth packages (library("sos"); findFn("{exponential smoothing}")).

Does that help?

Ben Bolker
  • 211,554
  • 25
  • 370
  • 453
  • thank you very very much for your very fast response! i am amazed by stackoverflow and you guys! i will try it and tell you about my results tomorow! – chameau13 Feb 13 '12 at 19:52
  • Despite its name, exponential smoothing is not a smoothing method but a one-sided filter used for forecasting. It sounds like chameau13 wants a true two-sided smoothing method. – Rob Hyndman Feb 13 '12 at 22:06
  • @RobHyndman: You're right about exponential smoothing (oops). You probably know more about this than I do. Do you have a better idea? I don't see how a two-sided smoothing method can have the property the OP wants ... ? – Ben Bolker Feb 13 '12 at 22:17
  • actually the method itself, or how it works is not so important. A one sided filter however sounds less susceptible to revisions. the reasosn I would need that: the time series at the moment is very volatile (it is an indicator which bases on a survey; the volatility problem is mostly due to our too small samplesize but I want to make it smoother and thus better interpretable). in the end i would like to use some seasonal decomposition, but at the moment this is not possble since I just have one year of data. – chameau13 Feb 14 '12 at 09:20
  • thank you very much for your help! today i finally found the time to test all your useful suggestions. for the solution I chose, see the answer above! thank you very much! – chameau13 Feb 19 '12 at 18:22
3

It seems you need a robust two-sided smoother. The problem is that an outlier at an end-point is indistinguishable from a sudden change in the trend. It only becomes clear that it is an outlier after several more observations are collected (and even then some strong assumptions of trend smoothness are required).

I think you will find it hard to do better than loess(), but other functions that aim to do robust smoothing include

  • smooth() for Tukey's smoothers;
  • supsmu() for Friedman's super smoother;

Hodrick-Prescott smoothing is not robust to outliers.

Rob Hyndman
  • 30,301
  • 7
  • 73
  • 85
  • thank you very much! i commented above without seeing you answer. sorry ! i will try all you suggestions tonight and report my results here. – chameau13 Feb 14 '12 at 09:26
  • thank you very much for your help! it took me a little longer to test your suggestions. however i ended up with a one sided smoother from the locfit package! merci beaucoup! – chameau13 Feb 19 '12 at 18:21