Running different programs on multiple cores

Question

Hopefully this is a simple question I've overlooked an answer to, but in my searches this morning I've been fruitless.

I have four cores on my computer, and want to concurrently run different programs (loops) in R over multiple cores. If I open up four R sessions, will they automatically be places on different cores, or do I need to figure out a way to do this with snowfall?

I have successfully used snowfall before to run the same program, but with different values before. But would snowfall allow me to run completely different programs on different cores? Could I set up each simulation loop to be a different function (i.e: func1, func2, func3, func4), and then have snowfall assign each function to a different core?

Thanks for any help

Hi Ben, unfortunately I can't do a proper test as I am in the middle of a fairly large simulation using up my cores. Trying to design my next simulation while the current one is running so I can be ready to go. Seems like I will follow Henrik's suggestion of just using multiple R sessions without needing snowfall — user1399311, Sep 27 '12 at 16:44

score 5 · Accepted Answer · answered Sep 27 '12 at 14:49

5

I don't know how it is on different OSs, but on windows your OS takes care of this. So you can open as many R sessions as you want and they will be distributed across the cores (with each process taking maximally one core). If you open more R sessions than you have CPUs, then obviously some will not be able to use a full CPU.

So no need to use snowfall here.

answered Sep 27 '12 at 14:49

Henrik

14,202
10
68
91

I don't understand the down vote. If you have completely different things to compute, `snowfall` or `parallel` is unnecessary and opening up different R consoles the way to go, as proposed by the OP. Distribution across cores is then done by the OS. – Henrik Sep 27 '12 at 15:09

score 3 · Answer 2 · answered Sep 27 '12 at 14:46

3

The easiest way to answer this question is:

Create four functions that each take a minute or two to run
Use snowfall
Look at your CPUs processor usage.

But to answer your question, yes they will be distributed across cores.

It might be easier to just use the parallel package that comes with R 2.15

answered Sep 27 '12 at 14:46

csgillespie

59,189
14
150
185

I'd always argue that `parallel` is easier because it a) comes with all recent versions and b) is effectively a rewrite / code review of the better parts of `snow` and `multicore`. – Dirk Eddelbuettel Sep 27 '12 at 15:01
The all recent versions is the clincher. – csgillespie Sep 27 '12 at 15:05

Running different programs on multiple cores

2 Answers2