Stack Overflow
Stack Overflow

Questions tagged [rparallel]

This tag refers to the `parallel` package, R core team. It provides support for parallel computation in R.

72 questions
2
votes
0 answers

Debug R parallel mclapply

It is difficult to debug the error code of mclapply because all values of a job are affected. I prepared a simple example. library(parallel) library(dplyr) data(iris) ## Parallel Version parFun <- function(i){ print(i) ## Generate a random…
Peter Pisher
  • 457
  • 2
  • 11
2
votes
0 answers

R parLapply taking increasing time to finish

Im running the function parLapply inside a loop and im verifying a strange behaviour. The time per iteration was increasing significantly and it didn't make much sense such an increase. So i started clocking the functions within the cycle to see…
user201305016
  • 41
  • 4
2
votes
0 answers

Can not makeCluster on windows 10

I can not use parallel::makeCluster or snow::makeCluster on a windows 10 computer. I frequently receive this error, each time with a different port number: Error in socketConnection("localhost", port = port, server = TRUE, blocking = TRUE, : …
Ahmed El-Gabbas
  • 398
  • 3
  • 10
2
votes
1 answer

parLapply: unused argument (MoreArgs

I was trying to debug a parLapply function that I am working on when I encountered an unexpected problem: the MoreArgs argument to parLapply is not working! It's 5AM at the moment, and I must be fatigued and missing something really obvious and…
Stonecraft
  • 860
  • 1
  • 12
  • 30
2
votes
1 answer

ggsave() does not work when called from mclapply() in RStudio; is there a work-around?

I would like to use the R 'parallel' library to parallelize generating some plots using the 'ggplot2' library and have run into a snag when attempting to do this within RStudio. Within the IDE, mclapply alternately freezes the session, or fails to…
James McIninch
  • 91
  • 3
2
votes
0 answers

Controlling the number of CPUs used by registerDoParallel

I have recently inherited a legacy R script that at some point trains a gradient boost model with a large regression matrix. This task is parallelised using the doParallel::registerDoParallel function. Originally, the script started the parallel…
Luís de Sousa
  • 5,765
  • 11
  • 49
  • 86
2
votes
1 answer

Error in parallel processing: port cannot be open

I am running different R scripts in batch mode at once in a linux cluster to estimate a model in different data sets (it also happens when I run it in Mac). The scripts are exactly the same, except for the data set that they are using. I get the…
Diogo
  • 842
  • 2
  • 11
  • 15
2
votes
2 answers

When (if ever) should I tell R parallel to not use all cores?

I've been using this code: library(parallel) cl <- makeCluster( detectCores() - 1) clusterCall(cl, function(){library(imager)}) then I have a wrapper function looking something like this: d <- matrix #Loading a batch of data into a matrix res <-…
Darren Cook
  • 27,837
  • 13
  • 117
  • 217
2
votes
1 answer

Max number of workers/slaves for parallel job snow

I'm running a foreach loop with the snow back-end on a windows machine. I have 8 cores to work with. The rscript is exectuted via a system call embedded in a python script, so there would be an active python instance too. Is there any benefit to…
imouellette81
  • 234
  • 2
  • 5
2
votes
1 answer

R parallel - parRapply not working properly

I'm doing some unit testing on a package in development. One of the tests is failing. Specifically, I have parallel versions of the code and non-parallel versions. The non-parallel version works perfectly. The parallel version fails a unit test,…
alexwhitworth
  • 4,839
  • 5
  • 32
  • 59
2
votes
0 answers

r-Error using foreach loop for parallel processing

I am trying to use 'doSNOW' for parallel processing in R. During the foreach loop command, it is showing error: object j is not found. Please help me in this regard. The code works fine with the normal for loop. But is takes like an eternity to…
Golam Rabbani Fahad
  • 23
  • 5
2
votes
1 answer

How to set up cluster slave nodes (on Windows)

I need to run thousands* of models on 15 machines (each of 4 cores), all Windows. I started to learn parallel, snow and snowfall packages and read a bunch of intro's, but they mainly focus on the setup of the master. There is only a little…
Tomas
  • 57,621
  • 49
  • 238
  • 373
1
vote
1 answer

Error in checkForRemoteErrors(val) : 7 nodes produced errors; first error: could not find function "fread"

All of the code included in this question is from the script called "LASSO code (Version for Antony)" in my GitHub Repo for this project. And you can run it on the file folder called "last 40" to verify my claim that it does run on limited sized…
Marlen
  • 171
  • 11
1
vote
1 answer

Passing objects via clusterExport or as function arguments

Is it more efficient to pass objects to parallel::parLapply and parallel::parLapplyLB as function arguments or to export them with parallel::clusterExport? I.e. parallel::parLapply(cl, 1:1000, function(y, x1, x2, x3, x4, x5) { ... }, x1, x2, x3,…
Chr
  • 1,017
  • 1
  • 8
  • 29
1
vote
1 answer

improve readability of output list of foreach loop

I have a simple example of foreach nested loop below. How can I improve the readability of the result res? Below is another example of nested for loop with more readable results, each element of list can be easily identified e.g.…
SiH
  • 1,378
  • 4
  • 18
1
2
3 4 5