Stack Overflow
Stack Overflow

Questions tagged [mclapply]

mclapply is a parallelized version of lapply, it returns a list of the same length as X, each element of which is the result of applying FUN to the corresponding element of X.

mclapply is a parallelized version of lapply. It returns a list of the same length as X, each element of which is the result of applying FUN to the corresponding element of X.

136 questions
2
votes
2 answers

How do I replace Quantstrat 'for loop' with mclapply [parallelized]?

I'd like to parallelize quantstrat. My code isn't exactly like this, but this showcases the issue. The problem I believe is the .blotter env is initialized to a pointer memory address and i am unable to initialize an array/matrix of…
thistleknot
  • 1,098
  • 16
  • 38
2
votes
2 answers

Reshape list created by lapply

I have vectors with latitudes and longitudes: longDim [1] -79.65770 -79.21761 -78.77750 latiDim [1] -39.70588 -39.26471 -38.82353 and I wanted to loop over their combination in parallel. To do so, first I used expand.grid to create a data frame…
thiagoveloso
  • 2,537
  • 3
  • 28
  • 57
2
votes
0 answers

BTYDplus: scheduled cores 1, 2, 8 did not deliver results, all values of the jobs will be affected

I'm using BTYDplus package functions. Sometimes, quite randomly I would encounter this error below Error in mcmc.list(lapply(draws, function(draw) draw$level_1[[i]])) : Arguments must be mcmc objects In addition: Warning message: In…
Afiq Johari
  • 1,372
  • 1
  • 15
  • 28
2
votes
1 answer

Snakemake for R pipeline -- SETUP

I am looking for some basic information about using Snakemake handle my R pipeline. It is my understanding that the two most common ways of doing this are by using the script flag and passing an R script, or using shell by passing to Rscript. If I…
abbas786
  • 401
  • 3
  • 11
2
votes
2 answers

parallel::mclapply() adds or removes bindings to the global environment. Which ones?

Why this matters For drake, I want users to be able to execute mclapply() calls within a locked global environment. The environment is locked for the sake of reproducibility. Without locking, data analysis pipelines could invalidate…
landau
  • 5,636
  • 1
  • 22
  • 50
2
votes
1 answer

mclapply encounters errors depending on core id?

I have a set of genes for which I need to calculate some coefficients in parallel. Coefficients are calculated inside GeneTo_GeneCoeffs_filtered that takes gene name as an input and returns the list of 2 data frames. Having 100-length gene_array I…
lizaveta
  • 353
  • 1
  • 13
2
votes
1 answer

Should mclapply calls be nested?

Is nesting parallel::mclapply calls a good idea? require(parallel) ans <- mclapply(1:3, function(x) mclapply(1:3, function(y) y * x)) unlist(ans) Outputs: [1] 1 2 3 2 4 6 3 6 9 So it's "working". But is it recommended for real compute-intensive…
dzeltzer
  • 990
  • 8
  • 28
2
votes
1 answer

Is mclapply() with mc.cores = 1 the same as lapply()?

I know that I should not nest parallel operators, but I am in a situation where I have to basically nest two mclapply() calls in my code. This is because in my code I have a function (let's call it foo()) that is already using mclapply() inside it.…
alghul96
  • 75
  • 5
2
votes
2 answers

mclapply with lme4 and long vectors

I am using mclapply from the parallel package to estimate mixed glmer models using the lme4 package on a high performance cluster. I am having the issue described here. I apply the suggested fix of adding mc.preschedule=F, but the problem persists.…
Erdne Htábrob
  • 819
  • 11
  • 29
2
votes
2 answers

Minimizing overhead with parallel functions in R

I tried reporting a bug I was running into in mclapply regarding large return values not being allowed. Apparently the bug has been fixed in development versions, but I'm more interested in the comment that the responder made: there was a 2GB limit…
James
  • 630
  • 1
  • 6
  • 15
2
votes
1 answer

mclapply vs parLapply speeds

I'm running on Linux and used mclapply easily. I run into some errors with parlapply, even after using clusterEvalQ. Before I go further to resolve the issue, is there any point, i.e. could there be a significant speed difference between the two or…
Oli
  • 532
  • 1
  • 5
  • 26
2
votes
0 answers

using multiple mclapply, parLapply

I am trying to chain multiple mclapply/parLapply: my_function<-function(x){ do something; mclapply() } Then I vary my x: mclapply(1:100,function(i)(function(x[i])),mc.cores=60) The problem is that I often get the following error: scheduled cores…
Vitalijs
  • 938
  • 7
  • 18
2
votes
2 answers

R error with mclapply in a foreach loop

Based on this post here, I tried to write a script, seen here: library(parallel) library(doParallel) cl<-makeCluster(2,outfile='') registerDoParallel(cl) foreach(i=1:5, .packages='parallel') %dopar% { system.time(mclapply(1:10,…
Plinth
  • 289
  • 1
  • 13
2
votes
1 answer

Replace apply function with lapply

I am creating a data set to compute the aggregate values for different combinations of words using regex. Each row has a unique regex value which I want to check against another dataset and find the number of times it appeared in it. The first…
HoneyBadger
  • 98
  • 2
  • 10
2
votes
1 answer

How do I convert "for" in "mclapply"?

I have this: # A simple "script" that receives an index ScriptR = function(i) { A = i^3; }; # With "for", pass the index "i" for "ScriptR" for (i in 1:10) { ScriptR(i); }; # How do I pass the index "i" for "ScriptR" function ?? …
Project Brots
  • 43
  • 2
1 2 3
…
9 10