Stack Overflow
Stack Overflow

Questions tagged [rparallel]

This tag refers to the `parallel` package, R core team. It provides support for parallel computation in R.

72 questions
0
votes
1 answer

R in Parallel - Partools Package - Error with `calm()` function

I am using the partools package to run linear regressions in parallel. I am doing this using the calm() function, which is a wrapper for the package's version of R's lm(). I'm using 20 cores on a 64gb node. I receive errors when I run the calm()…
Mason Malone
  • 166
  • 4
0
votes
0 answers

Simultaneous R sessions within different directories

I am looking in a way to start a new R instance working in a user-defined directory from a current R session. For example, let's say I have getwd() ## [1] "/Users/jplecavalier/projects/foo" and I want to do something…
J.P. Le Cavalier
  • 1,315
  • 7
  • 16
0
votes
1 answer

mclapply conflict with vecLib

I recently changed my R BLAS framework to vecLib, which ships with Mac. After doing so, I had problems with the parallel package in R. Here is an example: library(parallel) xx1<-matrix(runif(2*70),ncol=2) mcl.test<-mclapply(1:2,function(i)…
Devin F
  • 1
  • 2
0
votes
0 answers

Speed up rbindlist as .combine within foreach in R

Consider the following piece of code runing on a windows OS with doSNOW package: result.dt <- foreach(j = 1:nrow(keys), .combine = function(...) rbindlist(list(...)), .packages = c('data.table'), …
Chen Chen
  • 358
  • 4
  • 15
0
votes
1 answer

Multisession parallelism with a shared stderr redirect

I am trying to run R code with multisession parallelism such that all the error messages redirect to the same file. However, the sink() cannot be created. library(parallel) cl <- makePSOCKcluster(2) f <- function(){ …
landau
  • 5,636
  • 1
  • 22
  • 50
0
votes
0 answers

Why does mclapply function in R is more efficient than Rcpp + OpenMP?

I have a function with a loop (EstimateUniques) that is parallelized with OpenMP. I suggested that multithreading should be more efficient than multiprocessing, but when I compare this function with the simple run of "mclapply", it showed lower…
Viktor Petukhov
  • 41
  • 4
0
votes
1 answer

vectorizing & parallelizing the disagregation of a list

Here's some code that generates a list of data.frames and then converts that original list into a new list with each list element a list of the rows of each data frame. Eg. - l1 has length 10 and each element is a data.frame with 1000 rows. - l2…
alexwhitworth
  • 4,839
  • 5
  • 32
  • 59
0
votes
0 answers

Parallel computing for cross-validation with big.matrix in R

I am trying to use mclapply to parallelize cross-validation for modeling fitting procedure for very large design matrix X (~10GB) and response vector y. Let's say X is of dimension n-by-p: n=1000, p=1,000,000. Since X is very huge, it's backed as…
SixSigma
  • 2,808
  • 2
  • 18
  • 21
0
votes
1 answer

Possible to parallelize for loop with dependencies?

Hello veteran R users, I'm quite new to R and am wondering if there's any possibility of parallezing my process. My dataset is essentially derived from a pcap file where I've extracted the packets that correspond to a particular protocol-MODBUS/TCP.…
lstilo
  • 5
  • 1
  • 4
0
votes
0 answers

Performing Parallel computation using doSNOW and multiple servers in R

I am trying to do a multi-server (not multi-core) computation using doSNOW and foreach packages. I have 2 Windows servers and I want to start a parallel computation on both of these Windows machines. I have the following…
Sergei
  • 1,617
  • 15
  • 31
-1
votes
1 answer

Parallel computing in R: how to choose cores

In R, I using packages "doParallel" and "foreach" to parallel computing. Recently, I am running the program in HPC. It has four nodes, and each node has 16 processors. My R program only needs 4 processors. But the HPC always assigns the program to…
81235
  • 179
  • 2
  • 11
-3
votes
1 answer

Running parallel R on multiple hosts

Can you please provide a script to run parallel cluster on 2 hosts ( amazon ec2) from scratch on Ubuntu Linux Machine ? Specifications The hosts are connected from local machine via identity key named amazon_key.pem hostnames are supposed to be…
niths4u
  • 440
  • 3
  • 13
1 2 3 4
5