R package that is a “parallel backend” for the foreach package. It provides a mechanism needed to execute foreach loops in parallel.
Questions tagged [doparallel]
453 questions
3
votes
1 answer
Parallel computation in R for saving data over loops
My efforts in applying parallel on the below simple code to save outputs with Openxlsx over multiple loops is failed.
Anyone can help please to convert this code to the parallel mode. This code on real size data (over 50 million observations, takes…
Sean
- 103
- 9
3
votes
0 answers
Unable to load "foreach" library in RStudio
I'm going to work with 'doParallel' package, that requires 'foreach' package to be loaded. However, some strange error occurs with the error message copied below.
Yesterday I've updated R to 3.6.0 version. Could it be causing the…
Michał Stocki
- 31
- 3
3
votes
2 answers
Using standard R shiny progress bar in parallel foreach calculations
I am trying to use the standard R shiny progress bar in a parallel foreach loop using the doParallel back-end. However, this results in the following error message:
Warning: Error in {: task 1 failed - "'session' is not a ShinySession…
user1372987
- 197
- 1
- 9
3
votes
1 answer
foreach very slow with large number of values
I'm trying to use foreach to do parallel computations. It works fine if there are a small number of values to iterate over, but at some point it becomes incredibly slow. Here's a simple…
Rob Richmond
- 855
- 6
- 19
3
votes
1 answer
R Parallel processing error `Error in checkForRemoteErrors(val) : 6 nodes produced errors; first error: subscript out of bounds`
I'm learning parallel processing as a way to handle some huge datasets.
I have some variables predefined as below:
CV <- function(mean, sd) {(sd / mean) * 100}
distThreshold <- 5 # Distance threshold
CVThreshold <- 20 # CV threshold
LocalCV…
M八七
- 233
- 4
- 11
3
votes
1 answer
Scoping issue when using doParallel
I am trying to estimate multiple nonparametric models using the doParallel package. My problem though seems to be related to the np package.
Take a look at this reproducible example:
library(np)
library(doParallel)
df <- data.frame(Y =…
Martin Schmelzer
- 23,283
- 6
- 73
- 98
3
votes
1 answer
Parallel processed sentence generation creates garbled results
I am trying to create a dataset for some neural network learning purposes. Previously, I have used for loop to concatenate and make sentences but since the process was taking so long I implemented the sentence generation using foreach. The process…
Sooraj
- 514
- 4
- 20
3
votes
1 answer
error calling combine function loop foreach in R
The following code:
df <- foreach(i = 1:length, .combine = cbind) %dopar% {
...
...
m4
}
gives me an error :
error calling combine function:
giupardeb
- 791
- 1
- 5
- 13
3
votes
1 answer
Detect Cores from Multiple Nodes
I have a script in R that takes advantage of the doParallel package and the parallelized foreach function. I am currently registering my cluster by using a variation of the detectCores() command, which works quite well as the machine I am using has…
amelcher
- 61
- 1
- 7
3
votes
1 answer
Parallelise panel logit computations in R on a set of different explanatory variables
I'm a beginner to parallel computing in R. I came across the doParallel package and I thought it might be useful in my case.
The following code aims at evaluating in parallel several pglm regressions:
require("foreach")
require("doParallel")
resVar…
mike.dl
- 67
- 8
3
votes
1 answer
run h2o algorithms inside a foreach loop?
I naively thought it's straight forward to make multiple calls to h2o.gbm in parallel inside a foreach loop. But got a strange error.
Error in { :
task 3 failed - "java.lang.AssertionError: Can't unlock: Not locked!"
Codes below…
horaceT
- 621
- 13
- 26
3
votes
1 answer
Parallel foreach with two arguments
I have this code:
library(doParallel)
registerDoParallel(cores = 8)
result = foreach(A = c(1, 2, 3),B = c(10, 20), .combine = list) %dopar% {
A*B
}
result
[[1]]
[1] 10
[[2]]
[1] 40
but I want:
10, 20, 30, 20, 40, 60
user1436187
- 3,252
- 3
- 26
- 59
2
votes
0 answers
Best practice for providing optional automated parallel backend registration in an R package? (foreach)
I am writing an R package and would like to equip one of the functions with the option to run in parallel. Based on the advice from this response, I would like to give the user the option to either register their own parallel backend, or allow the…
Jacob Helwig
- 41
- 1
- 5
2
votes
0 answers
Run time parallel loop in R
I have access to a quite powerful remote desktop where I ran some parallel code in R. However, I am surprised about the results when I compare the running time between the parallel loop and the simple for loop. Below is my…
Dejan Evisal
- 21
- 1
2
votes
1 answer
Using doParallel with RMarkdown
I am trying to use doParallel with RMarkdown.
The R code calls a script called report.RMD. The goal is to produce 3 html reports from the iris dataset, each one named after the species with a table in that is filtered down to that species.
The…
Basil
- 747
- 7
- 18