0

I'm using R on R studio. In this case I can't use foreach. I would like to do something fairly simple: given a for loop: 

for(i:1:1000000)
{
#some code here 
}

One way to speed it up is to create two or more for loops and make them working on different chunks of data in different sessions.

Like:

for(i:1:500000)
{
 #some code here 
}

and

for(i:500001:1000000)
{
#some code here 
}

on two different sessions.

I would like to not to use the "system2" package. There must be a package that does it without having to manually open another session. How to do it?

Cheers,

Dario.

Dario Federici
  • 1,228
  • 2
  • 18
  • 40
  • 1
    look at the `foreach` package. BTW - asking for library recommendations is considered off topic here – dww Nov 14 '18 at 02:11
  • 1
    I mean - the `foreach` package and the `system2` package would both work, but you can't use them for unspecified reasons. What makes you think you'd be able to use any of the other packages that would let you do this? The [cran task view has a nice list of packages for parallel processing](https://cran.r-project.org/web/views/HighPerformanceComputing.html). – Gregor Thomas Nov 14 '18 at 03:05
  • I've tried foreach many times (I normally use it for work) but it does not work properly and when it works it does not speed up the loop. system2 is not appropriate in this case because I will have to deploy the script in a non suitable environment. – Dario Federici Nov 14 '18 at 03:21
  • 3
    Please provide some code. I've no idea what you want to do. Chunking + foreach is usually a good idea. See e.g. https://stackoverflow.com/a/42131766/6103040. – F. Privé Nov 14 '18 at 06:52

1 Answers1

0

I've found the solution in this package: "future.apply" that I find phenomenal.

It easily parallels the whole apply family.

I've used the "future_lapply" function and it works very well.

Dario Federici
  • 1,228
  • 2
  • 18
  • 40