2

I was wondering what is the difference between setting the number of cores for R to use via makePSOCKcluster and explictly in the foreach loop? Should I be setting this seperately in both instances, or is doing so when making the makePSOCKcluster enough?

cl <- makePSOCKcluster(max(1, detectCores() - 1))
registerDoParallel(cl)

and

mcoptions <- list(  preschedule=FALSE, mc.cores = max(1,detectCores()-1)  )
stream = foreach(i=1:NROW(stream_sponsored), .inorder=FALSE,
        .combine=rbind,
        .options.multicore=mcoptions)  %dopar% {
#do something
}
CorerMaximus
  • 653
  • 5
  • 15

1 Answers1

0

From [https://cran.r-project.org/web/packages/doParallel/vignettes/gettingstartedParallel.pdf][1]

'The “cores” options allows you to temporarily override the number of workers to use for a single foreach operation. This is more convenient than having to re-register doParallel. Although if no value of “cores” was specified when doParallel was registered, you can also change this value dynamically using the options function:

options(cores=2)
getDoParWorkers()
options(cores=3)
getDoParWorkers()

If you did specify the number of cores when registering doParallel, the “cores” option is ignored:

registerDoParallel(4)
options(cores=2)
getDoParWorkers()

As you can see, there are a number of options for controlling the number of workers to use with parallel, but the default behaviour usually does what you want.'

So, in short, if you are calling foreach once, then no need to specify cores inside foreach. But if you are calling two or more foreachs with different cores, then no need to specify cores in registerDoParallel

Vitali Avagyan
  • 1,193
  • 1
  • 7
  • 17