Cross Validation K-Fold with Forecast Package in R

Asked Nov 06 '17 at 21:51

Active Nov 06 '17 at 21:51

Viewed 550 times

I have created a model in R using the forecast package.

My source of learning this is from here: https://robjhyndman.com/hyndsight/dailydata/

I am using the last section which includes fourier series as such:

y <- ts(x, frequency=7)
z <- fourier(ts(x, frequency=365.25), K=5)
zf <- fourier(ts(x, frequency=365.25), K=5, h=100)
fit <- auto.arima(y, xreg=cbind(z,holiday), seasonal=FALSE)
fc <- forecast(fit, xreg=cbind(zf,holidayf), h=100)

After I create this model, is there a way I can do a cross validation k-fold test to determine the error and adjusted error?

I know how to do it with a generalized linear model as such:

library(boot)
lm1 <- glm(ValuePerSqFt ~ Units + SqFt + Boro, data = housing)
lm1cv <- cv.glm(housing, lm1, K=5)
lm1cv$delta
[1] 1870.31 1869.352

This shows the error and adjusted error.

Is there a function in the forecast package that can do this and it will help me compare the accuracy of this model with the glm model?

asked Nov 06 '17 at 21:51

nak5120

4,089
4
35
94

2

Standard CV makes little sense with time series models as you can only forecast the future based on the past. There are exceptions (see https://robjhyndman.com/publications/cv-time-series/), but this does not include dynamic regression models. – Rob Hyndman Nov 07 '17 at 05:48
@RobHyndman do you recommend I just use the accuracy function and compare errors on predictions? – nak5120 Nov 07 '17 at 13:23
standard cross-validation is pointless but I suspect @RobHyndman meant [time series cross validation](https://robjhyndman.com/hyndsight/tscv/), is it possible for ts regression models? – Kasia Kulma Apr 23 '18 at 15:03
I meant what I wrote. Read the link. – Rob Hyndman Apr 23 '18 at 22:01
How did you determine the value of K for your Fourier? – Ritesh Sinha Apr 16 '19 at 17:26
1

Answer is in this question here @RiteshSinha https://stackoverflow.com/questions/53198334/understanding-fourier-for-seasonality – nak5120 Apr 16 '19 at 21:28
This looks perfect !! – Ritesh Sinha Apr 18 '19 at 16:58
@RiteshSinha glad it helped! – nak5120 Apr 18 '19 at 23:47

Cross Validation K-Fold with Forecast Package in R

0 Answers0