Why is R for loop 10 times slower than when using foreach?

Question

This is really blowing my mind. The basic loop takes like 8 seconds on my computer:

system.time({
x <- 0
for (p in 1:2) {
    for (i in 1:500) {
        for (j in 1:5000) {
            x <- x + i * j
        }
    }
}
})
x

Whereas if I use foreach in non-parallel mode, it does take only 0.7 secs!!!

system.time({
x <- 0
foreach(p = 1:2, .combine = rbind) %do% 
    for (i in 1:500) {
        for (j in 1:5000) {
            x <- x + i * j
        }
    }
})
x

The result is the same, but foreach was somehow able to reach it much faster than basic R! Where is the inefficiency of basic R?

How is this possible?

In fact, I got complete opposite result compared to this one: Why is foreach() %do% sometimes slower than for?

This is a perfect example of how writing a package can improve upon/enhance base methods. — Rich Scriven, Jul 09 '14 at 10:51
@Richard great, please if you understand why and what happens then post an answer. — Tomas, Jul 09 '14 at 10:55
So if you take your triple-for-loop, make it into a function, and use `compile::cmpfun`, will the resultant function be as fast as `foreach` ? — Carl Witthoft, Jul 09 '14 at 11:39

James · Accepted Answer · 2014-07-09T12:55:20.637

foreach when used sequentially eventually uses compiler to produce compiled byte code using the non-exported functions make.codeBuf and cmp. You can use cmpfun to compile the innerloop into bytecode to simulate this and achieve a similar speedup.

f.original <- function() {
x <- 0
for (p in 1:2) {
    for (i in 1:500) {
        for (j in 1:5000) {
            x <- x + i * j
        }
    }
}
x
}

f.foreach <- function() {
x <- 0
foreach(p = 1:2, .combine = rbind) %do% 
    for (i in 1:500) {
        for (j in 1:5000) {
            x <- x + i * j
        }
    }
x
}

f.cmpfun <- function(x) {
f <- cmpfun(function(x) {
    for (i in 1:500) {
        for (j in 1:5000) {
            x <- x + i * j
            }
        }
        x
    })
    f(f(0))
}

Results

library(microbenchmark)
microbenchmark(f.original(),f.foreach(),f.cmpfun(), times=5)
Unit: milliseconds
         expr       min        lq    median        uq       max neval
 f.original() 4033.6114 4051.5422 4061.7211 4072.6700 4079.0338     5
  f.foreach()  426.0977  429.6853  434.0246  437.0178  447.9809     5
   f.cmpfun()  418.2016  427.9036  441.7873  444.1142  444.4260     5
all.equal(f.original(),f.foreach(),f.cmpfun())
[1] TRUE

Why is R for loop 10 times slower than when using foreach?

How is this possible?

1 Answers1

Linked