Stack Overflow
Stack Overflow

Questions tagged [doparallel]

R package that is a “parallel backend” for the foreach package. It provides a mechanism needed to execute foreach loops in parallel.

453 questions
0
votes
1 answer

How to make foreach %doparalell% execute all iterations?

I am setting up a foreach loop with %dopar%. The function inside should be executed n times. However no matter what the function is the function is executed only 6 times. There are no errors. How do I make foreach execute all iterations? I tried…
SKyJim
  • 111
  • 7
0
votes
1 answer

Why does "foreach" not export to my mutable object when running in parallel?

I'm setting up a patient level simulation model in R. It requires generating two data frames for each patient (with and without treatment) over time (using two inner loops). I then need to loop the inner loops for each patient that is required in…
JoeMoss
  • 31
  • 2
0
votes
1 answer

How to write foreach %dopar% logs to separate files

I use R to run Ant Colony Optimization and usually repeat the same optimization several times to cross-validate my results. I want to save time by running the processes in parallel with the foreach and doParallel packages. A reproducible example of…
achmed
  • 35
  • 4
0
votes
1 answer

Exporting a Parallel Foreach Call Into an Enviornment

I would like to export the out puts from foreach into an environment. I am pulling time series data from yahoo finance. library(quantmod) library(foreach) library(parallel) library(doParallel) registerDoParallel(cores=2) hub = new.env() tickers =…
Jordan Wrong
  • 1,205
  • 1
  • 12
  • 32
0
votes
1 answer

How to pass options to doparallel workers?

I use options() and getOption() to set options for how my code executes. In sequential operation it works fine. But when I use the doparallel package to run my code in parallel, the options don't get passed to the worker processes. Sample code that…
Andrew Schulman
  • 3,395
  • 1
  • 21
  • 23
0
votes
0 answers

doParallel computing using caret error message

I have been trying to implement multi-core processing with library(doParallel) but somehow one error kept showing up. I doubt it is an error generated from doParallel package. Below is my code (I skip the data processing part to make the code…
Nelson Chou
  • 21
  • 3
0
votes
1 answer

Running RSelenium in parallel using Docker

I am currently trying to use the package doParallel in order to parallelise my RSelenium web scraper (running on Docker). I have found this post (Speed up web scraping using multiplie Rselenium browsers) and am copying the answer provided by…
WillyWonka
  • 107
  • 1
  • 11
0
votes
0 answers

Parallelize matrix iteration

I'm playing with this dummy data to try to parallelize it, so far nI've not been able to achieve the same results as with the non-parallelized code. Dummy matrix: m <- matrix(1, nrow = 500, ncol = 500) m[1,1] <- 2 # To check the output keeps rows…
HeyHoLetsGo
  • 137
  • 1
  • 14
0
votes
1 answer

How to use available memory in R

I am running a 64-bit R/RStudio on a 64-bit Windows 10. Pc has 16GB of RAM and runs on 8-cores. So RStudio crashes at around 1.6/7 GB of memory utilization while reading a larger dataset. So I'm trying to use the parallel package to execute the…
Prometheus
  • 1,977
  • 3
  • 30
  • 57
0
votes
0 answers

R future, doSNOW, parallel and foreach for asynchronous execution

I'm struggling with choosing proper package for parallel asynchronous execution of scripts by external programs with R (R script also provides inputs and aggregates outputs), I know about four popular packages : future, doSNOW, parallel and foreach…
Qbik
  • 5,885
  • 14
  • 62
  • 93
0
votes
0 answers

parSapply on a dataset (Running a function on multiple cores for each row of a dataset)

This question is specifically about the use of multiple cores to run a given function where the function requires a package and additional arguments to run. I have a large dataset of the following form: Event_ID =…
Prometheus
  • 673
  • 3
  • 25
0
votes
1 answer

JVM Error While Writing Data Frame to Oracle Database using parLapply

I want to parallelize my data writing process. I am writing a data frame to Oracle Database. This data has 4 million rows and 8 columns. It takes 6.5 hours without parallelizing. When I try to go parallel, I get the error Error in…
boyaronur
  • 521
  • 6
  • 18
0
votes
2 answers

Issue with doParallel and foreach. I can register cores, but they don't seem to run

I have a large data set (about 40 million rows x 4 columns) and I want to perform a Fisher test on the data in each row. An example of the data looks like this: refAppleBase altAppleBase refHawBase altHawBase 1 115 1 94…
Mac
  • 1
  • 1
0
votes
0 answers

Assignments within foreach and saving output from foreach iteratively

I am connecting to a local SQL Server database and running some queries in loop. The output from each iteration is saved as a RDS datafile locally on the disk. This works fine when I execute this sequentially in standard for loop as shown…
Science11
  • 788
  • 1
  • 8
  • 24
0
votes
0 answers

R: poor performance of doParallel to manipulate large dataframes

first of all thank you for taking the time to read my question. More than anything, I need conceptual help, since I do not understand what is wrong with my interpretation. Some time ago I try to refactor some of the algorithms that I use, so that…
Leonel Esteban Bracco
  • 21
  • 4
1 2 3
…
30 31