1

I am using R 2.14.0 64 bit on Linux. I went ahead and used the example described here. I am then running the example -

library(doMC)
registerDoMC()
system.time({
r <- foreach(icount(trials), .combine=cbind) %dopar% {
  ind <- sample(100, 100, replace=TRUE)
  result1 <- glm(x[ind,2]~x[ind,1], family=binomial(logit))
  coefficients(result1)
} })

However, I see in top that it is using only one CPU core. To prove it another way, if I check a process that uses all cores, I see -

ignorant@mybox: ~/R$ ps -p 5369 -L -o pid,tid,psr,pcpu
  PID   TID PSR %CPU
 5369  5369   0  0.1
 5369  5371   1  0.0
 5369  5372   2  0.0
 5369  5373   3  0.0
 5369  5374   4  0.0
 5369  5375   5  0.0
 5369  5376   6  0.0
 5369  5377   7  0.0

But in this case, I see -

ignorant@mybox: ~/R$ ps -p 7988 -L -o pid,tid,psr,pcpu
  PID   TID PSR %CPU
 7988  7988   0 19.9
ignorant@mybox: ~/R$ ps -p 7991 -L -o pid,tid,psr,pcpu
  PID   TID PSR %CPU
 7991  7991   0 19.9

How can I get it to use multiple cores? I am using multicore instead of doSMP or something else, because I do not want to have copies of my data for each process.

ignorant
  • 1,390
  • 1
  • 10
  • 14
  • What's using 1 core? The original R session? Look for additional instances of `rsession` instead. From the first page in the linked document, `... the multicore functionality starts its workers using fork without doing a subsequent exec ...`. `fork` creates a new process, not a new thread within the existing process. – Matthew Lundberg Apr 03 '13 at 22:11
  • This is a possible duplicate of http://stackoverflow.com/questions/12924698/parallel-processing-in-r-limited – cbeleites unhappy with SX Apr 06 '13 at 23:16
  • @cbeleites: I agree. In fact, I flagged it as duplicate several days ago. – Steve Weston Apr 06 '13 at 23:19

2 Answers2

4

First, you might want to look at htop, which is probably available for your distribution. You can clearly see the usage for each CPU.

Second, have you tried setting the number of cores on the machine directly?

Run this with htop open:

library(doMC)
registerDoMC(cores=12) # Try setting this appropriately.
system.time({
  r <- foreach(1:1000, .combine=cbind) %dopar% {
    mean(rnorm(100000))
  } })
# I get:
#    user  system elapsed 
#  12.789   1.136   1.860

If the user time is much higher than elapsed (not always -- I know, but a rule of thumb), you are probably using more than one core.

nograpes
  • 18,623
  • 1
  • 44
  • 67
  • 1
    @MatthewLundberg Nice. I do find that `htop` is a little nicer-looking. – nograpes Apr 03 '13 at 23:04
  • tried this, but sadly still only one core is being used. I used top -> 1 to verify and also ps. I know I can use snow, etc, but then it will make copies of data, which is very expensive in my case – ignorant Apr 04 '13 at 22:23
4

You could try executing your script using the command:

$ taskset 0xffff R --slave -f parglm.R

If this fixes the problem, then you may have a version of R that was built with OpenBLAS or GotoBlas2 which sets the CPU affinity so that you can only use one core, which is a known problem.

If you want to run your example interactively, start R with:

$ taskset 0xffff R
Steve Weston
  • 19,197
  • 4
  • 59
  • 75