Stack Overflow
Stack Overflow

Questions tagged [doparallel]

R package that is a “parallel backend” for the foreach package. It provides a mechanism needed to execute foreach loops in parallel.

453 questions
0
votes
1 answer

Using foreach in a function definition vs. using foreach to write to a file

I'm having difficulty parallelizing R for optimizing the speed of function calls when writing to a file. The function is simple, but the files it creates are massive, and they take an unreasonable amount of time. I've been using profvis to visualize…
user8173816
  • 55
  • 4
0
votes
1 answer

doParallel seems to corrupts output for unknown reason

I'm attempting to run the following code (used for EWAS purposes) in a parallel format: registerDoParallel(2) combined <- foreach(i = 28:78, .combine=rbind) %dopar% { mm<-rep(NA,412) FIDunique<-unique(pheno$FID) for(j in…
Aiden
  • 1
0
votes
0 answers

Repeated iterations in foreach loop

I am using the foreach(i = 2:366) function in R with doParallel() as my back-end. In the past I haven't had any issues but now I'm attempting to run 365 complex iterations in parallel and some are being repeated. As the foreach() output rbinds the…
Dustin Roten
  • 73
  • 3
0
votes
1 answer

write to file in foreach %dopar% loop

I have this code which I want to make parallel however I can't seem to get it to work. The idea is that for each value chr, snp_sel(geno_data,k, bl) gives me back a matrix of k columns these columns are subsequently written one by one to a file.…
jjm
  • 21
  • 4
0
votes
1 answer

Nested do.call within a foreach %dopar% environment can't find function passed with .export

I am nesting multiple levels of do.call (each themselves using functions named in the parameters, not hard-coded) within a %dopar% parallelized environment, and a function from my outside environment can't be found by the innermost function. I know…
Randall
  • 3
  • 1
0
votes
1 answer

R foreach doParallel with 1 worker/thread

Well, I don't think anyone understood the question... I have a dynamic script. Sometimes, it will iterate through a list of 10 things, and sometimes it will only iterate through 1 thing. I want to use foreach to run the script in parallel when the…
Pablo Boswell
  • 805
  • 3
  • 13
  • 30
0
votes
1 answer

Opening image files in R in parallel

What I'm trying to do: Open a stack of images using EBImage, process them, and save the processed image in a new file. I am attempting this in parallel using the package "doParallel" and "foreach". The problem: Any time I use more than one processor…
J S
  • 75
  • 1
  • 5
0
votes
0 answers

Error while running lmer with 'foreach' and 'dopar' in R

I'm trying to run mixed models using 'foreach' and 'dopar' but I keep getting an error that says "task 1 failed - "could not find function "lmer"". Here's an example: library(lme4) A=c(30,39,52,58,90,102,109,120) B <-…
sh1291
  • 45
  • 1
  • 1
  • 5
0
votes
1 answer

cv.glmnet parallel and memory issue

It is the first time I am using parallel processing in general. The question is mainly about my poor syntax. I would like some help in capturing the output for a large number of cv.glmnet iterations, as I believe I have built cv_loop_run to be badly…
J. Doe.
  • 1,255
  • 1
  • 12
  • 25
0
votes
1 answer

Efficiently and selectively combining columns in R

I have the following data countrycols = alljson[,c("country_gc_str","country_ipapi_str","country_tm_str")] head(countrycols) country_gc_str country_ipapi_str country_tm_str 1 RU RU 2 …
heights1976
  • 480
  • 6
  • 16
0
votes
1 answer

Nested foreach and dopar - bootstrapping each row of data frame

I have data frames that look similar to this: maindata <- data.frame(cbind(num=c(79,61,62,57), denom=c(162356,170189,164634,162006), group=c(1,2,3,4))) My intention is to select each row, perform…
ajg0101
  • 3
  • 2
0
votes
1 answer

Possible reasons for " Error in checkForRemoteErrors(lapply(cl, recvResult)) : ... object '.doSnowGlobals' not found" Error?

I have been executing a function script repeatedly in R for many years. Within the function definition, I set up a parallel cluster using on my multi-core Windows workstation using: # cores0 <- 20 (cores set to 20 outside of function definition) …
KDA
  • 311
  • 2
  • 12
0
votes
2 answers

Parallel upper triangle dataframe (matrix) with missing value in R

I have a vector a with missing value (the missing value is 5 and there might be more missing values). I would like to create upper triangle dataframe (matrix) s in parallel. The final dataframe s should be that is based upon a index on b: > s 1 …
Avi
  • 2,247
  • 4
  • 30
  • 52
0
votes
0 answers

multicombine in parallel processing in R

I use the following code: library(foreach) library(doParallel) N<-5 cl<-makeCluster(4) registerDoParallel(cl) comb <- function(x, ...) { lapply(seq_along(x), function(i) c(x[[i]], lapply(list(...), function(y) y[[i]]))) …
Avi
  • 2,247
  • 4
  • 30
  • 52
0
votes
0 answers

Time estimation of parallel process of nested loops

I have the following code: library(foreach) library(doParallel) N<-5 cl<-makeCluster(4) registerDoParallel(cl) start.time <- Sys.time() #loop #result is a list of vectors s8 <- foreach(i=1:N) %:% foreach(j=i:N, .combine='c')…
Avi
  • 2,247
  • 4
  • 30
  • 52
1 2 3
…
30 31