0

I'm doing a rolling regression analysis and wanted to improve execution speed using multiple cores of my PC. The code i tried looks like this:

library(doParallel)
y <- reg.data[,1][1:15000]
x1 <- reg.data[,2][1:15000]
x2 <- reg.data[,3][1:15000]
windowSize = 8820
registerDoParallel(cores=6)
ptm <- proc.time()
z1 <- foreach(i=seq(1, (length(y)-windowSize+1), 1), .combine=rbind) %do% {
  idx <- i:(i+windowSize-1)
  coefficients(lm(y[idx]~0+x1[idx]+x2[idx]))
}
print(proc.time() - ptm)

The code executes and produces correct results. The problem is that no matter what i am doing i get roughly the the same result in terms of time spent, if it is 2 cores or 6 i've registered with registerDoParallel(). Which is in sharp contrast to what many users have reported on similar tasks. Looking at TaskManager in Windows i can see the that all 6 cores are doing something and there is virtually no speedup compared to 2 cores (less that a second on a total ~ 60 seconds execution time). Any ideas what i am doing wrong?

flipper
  • 531
  • 4
  • 13
  • https://pbs.twimg.com/media/CAmkvqKWsAAhuI8.jpg – Khashaa Mar 25 '15 at 07:46
  • Hahaha nice pic @Khashaa – Dominic Comtois Mar 25 '15 at 07:51
  • If you only need coefficients, using `fastLm` from `RcppArmadillo` speeds the things up significantly. – Khashaa Mar 25 '15 at 07:53
  • If Kjashaa's solution is not enough and you want to try another package, I've had decent results with doSNOW... also maybe see this link: http://comments.gmane.org/gmane.comp.lang.r.hpc/1396 – Dominic Comtois Mar 25 '15 at 07:57
  • thanks. i'll check fastLm out and doSNOW. – flipper Mar 25 '15 at 07:59
  • I would use [mclapply](https://stat.ethz.ch/R-manual/R-devel/library/parallel/html/mclapply.html) with `mc.preschedule = TRUE` (or another alternative that allows pre-scheduling and possibly even runs on Windows or even do the pre-scheduling manually) in order to minimize overhead. I would also use `lm.fit` for the OLS. – Roland Mar 25 '15 at 08:27
  • 2
    Shouldn't you use `%dopar%` instead of `%do%`? – cryo111 Mar 25 '15 at 17:15
  • cryo111 thanks, the problem was with do instead of dopar. massive speed up now... – flipper Mar 26 '15 at 10:31

0 Answers0