0

Today I was trying to use Theil's U2 from DescTools instead of forecast package. I am just wondering, why are both functions returning different results? As far as I am informed, there should be no difference.

library(forecast)
library(DescTools)

fc_df = data.frame(
fc1 = c(5565,5448,5164,5067,4997,5035,5168,5088,5162,4990,5018,5782),
fc2 = c(2565,2448,2164,2067,1997,1035,2168,2088,2162,1990,1018,2782)
)
act_df = data.frame(
act1 = c(9370,7980,6050,5640,6220,5740,6040,5130,5090,5210,4910,6890),
act2 = c(2900,2160,2400,2020,1630,1660,2210,1930,1960,1590,1730,2440)
)
# forecast 
ts_act <- ts(act_df, frequency = 12)
do.call(what = rbind, args = lapply(1:ncol(fc_df), function(x){
                               forecast::accuracy(fc_df[, x], ts_act[, x])}
                              ))
# DescTools ts 
TheilU(fc_df$fc1, ts_act[, 1])
TheilU(fc_df$fc2, ts_act[, 2])
Leonhard Geisler
  • 506
  • 3
  • 15

1 Answers1

1

Unfortunately, there are several statistics known as "Theil's U", partly because Theil himself used the same notation for different statistics in different papers.

Suppose the forecasts are stored in the vector f and the actuals are stored in the vector a, each of length n. Then the forecast package is returning a statistic based on relative changes.

fpe <- f[2:n]/a[1:(n-1)] - 1
ape <- a[2:n]/a[1:(n-1)] - 1
theil <- sqrt(sum((fpe - ape)^2)/sum(ape^2))

The DescTools package returns two types of Theil's U statistic. type=2 is

theil <- sqrt(sum((f-a)^2)/sum(a^2))

while type=1 is given by

theil <- sqrt(sum((f-a)^2/n))/(sqrt(sum(f^2)/n) + sqrt(sum(f^2)/n))
Rob Hyndman
  • 30,301
  • 7
  • 73
  • 85
  • Thanks a lot! Now I get the difference. Do you got any suggestions on scientific work on Theil's U2 related to time series? It is still quite exotic in time forecasting practice, even though it is in my opinion more robust and meaningful than the popular measures. I mainly want to use it as an accuracy measure for ex ante forecasts and therefore need some good arguments to sell it to customers, that may be reluctant to learn the math behind it – Leonhard Geisler Mar 01 '22 at 09:06