4

I'm trying to use DEoptim optimization package in R on a continuous optimization problem, and as my cost function takes a long time to evaluate (2min), I'm trying to use parallel computation. My questions are:

  1. What's the difference between paralleltype=1 (parallel) and 2 (foreach) option ? When to use one of the two ?

  2. Is it possible to specify the number of cores with paralleltype=1, in order not to take all the available cores for computing (50 cores on 64 cores available, for instance)?

Mathieu
  • 43
  • 3

2 Answers2

1

ad. 1 - It is explained in the package documentation.

ad. 2 - Requires some tinkering with two functions: DEoptim.control and DEoptim

First, add a variable - say limitCores - to DEoptim.control function, which controls execution parameters of the DE optimization.

Second, introduce modification to the DEoptim wrapper function to act on the limitation set by limitCores.

if (ctrl$parallelType == 1) {

    if (!is.na(ctrl$limitCores)) {

        if (ctrl$limitCores<1) useCores <- round(parallel::detectCores()*ctrl$limitCores) else useCores <- ctrl$limitCores

        cl <- parallel::makeCluster(parallel::detectCores())

    } else {

        cl <- parallel::makeCluster(parallel::detectCores())

    }

Complete code: http://pastebin.com/NumDx4ae

mjaniec
  • 1,074
  • 1
  • 9
  • 12
  • Thanks. But I do not see anywhere in the package documentation when to use one type of parallelism instead of another. I do see that paralleltype=1 uses parallel package and =2 uses foreach package, but that's it. – Mathieu Jul 13 '15 at 08:05
  • 1
    parallelType points to parall backend that will be used. Some parallel backends are system specific and do not work on some systems. Currently the parallel processing in R is pretty solid. A couple of years ago it was not so, and choosing the right backend was critical. – mjaniec Jul 13 '15 at 20:44
  • Probably a bit outdated, but may I ask whether it shouldn't be something essentially using useCores, because in the version above my computer uses detectCores() if I use limitCores>1 – Stefan Voigt Jul 22 '16 at 07:37
  • My code at Pastebin is one year old, but as far as I remember, the general idea was to limit the PROPORTION of the used cores / threads. So, >=1 means ALL the cores / threads should be involved. – mjaniec Jul 25 '16 at 04:38
0

Thank you for your code mjaniec!

I know it has been a while since you posted it, but I think in line 91 it should be: 

cl <- parallel::makeCluster(useCores)

instead of:

cl <- parallel::makeCluster(parallel::detectCores())
Anika
  • 1
  • 1