Stack Overflow
Stack Overflow

Questions tagged [mclapply]

mclapply is a parallelized version of lapply, it returns a list of the same length as X, each element of which is the result of applying FUN to the corresponding element of X.

mclapply is a parallelized version of lapply. It returns a list of the same length as X, each element of which is the result of applying FUN to the corresponding element of X.

136 questions
1
vote
0 answers

IRkernel dies when running mclapply with dada2

I don't know if this issue is dada2 specific or not. I would guess that it is not but I am not able to reproduce it otherwise. I am trying to use mclapply from the parallel library inside of a Jupyter notebook with dada2. The parallel job runs,…
johnchase
  • 13,155
  • 6
  • 38
  • 64
1
vote
0 answers

expand.grid - try to solve "cannot allocate vector of size" issue

I need to create huge data.frame of combinations, but I don't need them all. But as I saw here, expand.grid function is not able to add specific condition which combination throw out. So I decided to go step by step. For example I have variants<-9…
Bury
  • 527
  • 2
  • 5
  • 15
1
vote
0 answers

mclapply in R: object 'X' not found

I'm trying to use mclapply from the parallel package (on a Mac) and am running into an issue. I've defined a function edge.loop as: edge.loop <- function(index) { hw.edge <- E(G)[HW.edges][index] apath <- E(G)$emig.route[[hw.edge]] …
wonderboy3489
  • 33
  • 3
1
vote
0 answers

Performance overhead with R function mclapply

I am experiencing performance overhead while using R function mclapply on quite large datasets. For instance, let's consider the following toy example (it consists in suming the elements of sub-matrices in parallel): n = 30000 X =…
Vincent M
  • 11
  • 2
1
vote
1 answer

improper use case of mclapply?

I've been playing with rstudio server on an old laptop with linux mint (debian) installed. I've always ran on windows, so I've never taken advantage of parallel or multicore packages, and my goal was to learn rstudio server as well as R linux and…
Andrew Taylor
  • 3,438
  • 1
  • 26
  • 47
1
vote
1 answer

Fuzzy matching between two lists - improving double loop for efficiency

I am looking for some advice on the best reworked implementation of the objective of the below MWE that effectively has to check each element in a list against each element in another list using agrep; this example is 2x2 but my actual problem is…
SOConnell
  • 793
  • 1
  • 8
  • 27
1
vote
1 answer

Application of mclapply() to a function writing to a global variable

I'm trying to use parallel::mclapply to speed up the calculation of the following code: library(raster) library(HistogramTools)#for AddHistogram #Create a first h here for the first band... omitted for brevity readNhist <- function(n,mconst) { l…
AF7
  • 3,160
  • 28
  • 63
1
vote
1 answer

How to pass function arguments when creating a list of tasks in R?

I'm trying to create a list of tasks to run in R parallel using mclapply. library(parallel) tasks <- list( job1 = y(5), job2 = y(6) ) # Using fork() out <- mclapply( tasks, function(f) f(), mc.cores = length(tasks) ) where y is for…
DaReal
  • 597
  • 3
  • 10
  • 24
1
vote
0 answers

tm_map has parallel::mclapply error in R 3.0.1 on Linux

I am using R 3.0.1 on Platform: i486-pc-linux-gnu (32-bit).I am trying to use tm_map from the tm library.I have 4080 words in my SmartStopWord list. But when I execute the this code : library(tm) Get_Corpus <- system.file("texts","MyFile", package…
Vampir
  • 21
  • 5
1
vote
1 answer

Efficient way to add-up all numbers except the ones that accompany an I using mclapply

I have the following vector: my.vector = c("4M1D5M15I1D10M", "3M", "4M2I3D") And I'd like to transform it into the following vector: my.result = c("21N", "3N", "7N") The logic for such results is as follows, for "4M1D5M15I1D10M" I added all the…
Dnaiel
  • 7,622
  • 23
  • 67
  • 126
1
vote
1 answer

Efficient way to split a vector of cigars using mclapply

I have a very large vector of cigars: my.vector = c("44M2D1I","32M465N3M", "3S4I3D45N65M") That I'd like to transform to a vector of splitted cigars - the logic is as follows: whenever I find a number followed by an N, I have to split it, that is…
Dnaiel
  • 7,622
  • 23
  • 67
  • 126
1
vote
1 answer

Can mclapply work with a CompressedRleList?

Can mclapply work with a CompressedRleList? For example, I have a vector of cigars (a), and the cigarToRleList returns CompressedRleList given that vector: a = c("44M","44M","9S35M","44M","40M4S") b = cigarToRleList(a) If I try to run mclapply on…
user1701545
  • 5,706
  • 14
  • 49
  • 80
1
vote
1 answer

Why is mclappy slower than apply in this case?

i'm pretty confused. I want to speed up my algorithm by using mclapply:parallel, but when I compare time efficiency, apply still wins. I'm smoothing log2ratio data by rq.fit.fnb:quantreg which is called by my function quantsm and I'm wrapping my…
user1991825
  • 309
  • 1
  • 5
  • 16
1
vote
1 answer

Creating an R function to use mclapply from the multicore package

I need to analyze some simulated data with the following structure: h c x1 y1 x1c10 1 0 37.607056431 104.83097593 5 1 1 27.615251557 140.85532974 10 1 0 34.68915314 114.59312842 2 1 1 …
Ashley Naimi
  • 232
  • 1
  • 9
0
votes
1 answer

mclapply() instead of nested for loops

I have a huge matrix and would like to calculate the distance matrix using pairwise comparison of all columns. I know that built in dist() functioin in R already does the job but I would like to do this manually using nested for loops. I have…
say.ff
  • 373
  • 1
  • 7
  • 21
1 2 3
…
9 10