Stack Overflow
Stack Overflow

Questions tagged [doparallel]

R package that is a “parallel backend” for the foreach package. It provides a mechanism needed to execute foreach loops in parallel.

453 questions
0
votes
0 answers

caret package not using all the cores

I am running a model on an AWS instance with 36 cores. Dummy data example: library(mlbench) data(Sonar) library(caret) library(doParallel) set.seed(95014) # create training & testing data sets inTraining <- createDataPartition(Sonar$Class, p =…
Hanjo Odendaal
  • 1,395
  • 2
  • 13
  • 32
0
votes
2 answers

doParallel performance on a tensor in R

I need to perform some operations on a tensor and I would like make this parallel. Consider the following example: # first part without doParallel N = 8192 M = 128 F = 64 ma <- function(x,n=5){filter(x,rep(1/n,n), sides=2)} m <-…
learning-man
  • 119
  • 2
  • 11
0
votes
0 answers

R - Utilize each of the CPU (Not Parallel processing)

I recently switched to AWS EC2 to do my data analytics using Ubuntu R Studio. I believe it will provide me with more computing power than my MacBook. I am looking for ways to better utilize the computing power (even under the t2.micro instance which…
Bosco Lam
  • 43
  • 2
0
votes
0 answers

In R huge difference between user and elapsed which is not equal to "system"

I am running a script which uses 3 of the 4 cores of my windows machine using the function %dopar%. I get the following statistics: user system elapsed 1.19 0.06 75.83 Without the parallel utilization (i.e. using only one core), user and…
Omry Atia
  • 2,411
  • 2
  • 14
  • 27
0
votes
0 answers

Running for loops within foreach loop

I'm trying to run a simulation with parameter values drawn from different distributions for many times. And I would like to save the results from each run by column to different matrices. This should be pretty common but I have a hard time…
CYH
  • 43
  • 5
0
votes
0 answers

doParallel- foreach in R not processing results consistently for every iteration

I am running a large loop in parallel to do regression. The loop was working fine and everything was well but I am suddenly observing very strange a behavior. Some iterations do not reach the step of regression itself. Here's a simple snippet of…
user3825354
  • 13
  • 6
0
votes
1 answer

Using ForEach to Step Through columns for 1000's of regressions

First some data. Make a dataframe for covariates and my outcome of interest for regression and one for explanatory variables. What I am doing is stepping through the lm(outcome ~ mycovs + ith column of betas) and for this example, collecting the…
akaDrHouse
  • 2,190
  • 2
  • 20
  • 29
0
votes
0 answers

Optimize dopar R

I want to loop over 116 groups (i) to do the follow: r<- foreach(i=unique(Error$scidx),.combine = c) %dopar% { numerator=sum(P1new[Error$scidx==i])+sum(P2new[DATA$scid==i]) denominator=sum(P1old[Error$scidx==i])+sum(P2old[DATA$scid==i]) …
Bobby W Chung
  • 13
  • 4
0
votes
1 answer

R package gamlss used inside foreach() fails to find an object

I'm trying to get leave-one-out predicted values. Please help me with this "can't find object" issue. I have searched for similar issues, but haven't managed to figure it out. This is on Windows 10. Thanks in…
James
  • 237
  • 1
  • 12
0
votes
0 answers

Parallelized estimation of glarma time series models in R with parLapply instable

I work with glarma (generalized autoregressive moving average) time series models in R, provided by the package glarma. Since a lot of independent estimations are performed on similar data sets I parallelized the procedure via parLapply.…
7shoe
  • 1,438
  • 1
  • 8
  • 12
0
votes
0 answers

Rscript calling different R version

I have read this post and this post and this post but the solutions there (most notably, using the full path to Rscript) have not resolved my problem: I have installed the latest R on a server in my local directory (because the system wide R is out…
Eric
  • 1,691
  • 16
  • 24
0
votes
1 answer

R foreach %dopar% Results

I am trying to run a function using foreach and %dopar% that will pass its results back into itself for each iteration. Small example below: require(doParallel) test_function <- function(data) { result <- rbind(data, data) …
93i7hdjb
  • 1,136
  • 1
  • 9
  • 15
0
votes
0 answers

data.table operations with %dopar% are very slow

I run a loop over elements of list grouped_data_list using foreach and dopar. The runtime is terribly slow, while workers are visibly busy. If I make a vectorized routine with lapply, and without parallelling, this takes seconds. What is wrong with…
Alexey Burnakov
  • 259
  • 2
  • 14
0
votes
0 answers

Calling Fortran subroutines from R foreach on Linux platform

rm(list=ls()) library(doParallel) dyn.load("bar.so") foo <- function(x) { if (!is.numeric(x)) stop("arg x must be numeric") out <- .Fortran("bar", n=as.integer(length(x)), …
Yvonne Collingham
  • 1
0
votes
2 answers

R - doParallel does not work with Student and Fisher tests

So I am using the R package doParallel to parallelized some steps of my script when I have to handle large list of elements to compute it faster. Since this time all of the functions I used so far were wroking perfectly well with foreach() : I just…
Yoann Pageaud
  • 412
  • 5
  • 22
1 2 3
…
30 31