Use this tag for questions specific to the future ecosystem in R - typically used for parallel and asynchronous processing. At the core of this ecosystem is Future API and the future package, which a larger set of packages then build upon.
Questions tagged [r-future]
94 questions
0
votes
0 answers
How to run reactive background process in shiny R?
I've written code that runs a long calculation in which, as part of it, several UI elements are updated showing part of the progress and results of the calculations. I would like to make possible for the user to run more than one calculation at the…
David Díaz
- 141
- 2
- 3
- 16
0
votes
0 answers
setDT() in functions fails when using future_lapply
Possibly related to this question.
future_lapply throws an error when using setDT (or as.data.table for that matter) is used inside a function, complaining that the object is not a data.table.
This only happens when using the
tweaking multisession:…
vinceroni
- 87
- 5
0
votes
1 answer
When performing multisession work, future_lapply says that a package doesn't exist, but it works fine when running plan(sequential)
When I try to use future_apply with plan(multisession), it says that the package I'm trying to use doesn't exist. When I use plan(sequential) it works fine. I also get the same error when using plan(callr).
Here's the error:
Error in…
Roger-123
- 2,232
- 1
- 13
- 33
0
votes
1 answer
How to parallelize future_pmap() across multiple slurm nodes
I have access to a large computing cluster with many nodes each of which has >16 cores, running Slurm 20.11.3. I want to run a job in parallel using furrr::future_pmap(). I can parallelize across multiple cores on a single node but I have not been…
qdread
- 3,389
- 19
- 36
0
votes
1 answer
R future multisession Error: values() is defunct in future (>= 1.20.0). Use value() instead
I had a function successfully using future multisession, but with an update of future received the below error, which I have not figured out how to solve:
Error: values() is defunct in future (>= 1.20.0). Use value() instead.
Please see below…
Oscar Kjell
- 1,599
- 10
- 32
0
votes
0 answers
R: parallelize linear regression
I'm running linear regression on COVID-19 data over all 3000+ US counties and the code is running pretty slow. Are there options to parallelize this?
I've tried furrr::future_map() but it doesn't really speed up the process that much. CPU usage is…
Rob Hanssen
- 153
- 8
0
votes
1 answer
Use parallel processes for API Call use future
My shiny app displays 10 different phovotvoltaic systems. For every pv-system the app does the exact same calculations. I use modules in shiny to reduce the amount of lines. The process is like follwing:
The data is loaded from an API (Module)
The…
jogabonito
- 7
- 1
0
votes
0 answers
How to run parallel GRASS GIS mapsets from R using rgrass7 in a drake/targets pipeline?
I want to parallelize my GRASS GIS analysis and calculations using the package rgrass7 from within a targets pipeline. As I am a newbie to GRASS, I am not sure how to correctly set up multiple mapsets in the same GRASSGIS database. So far I tried…
MxNl
- 371
- 2
- 9
0
votes
1 answer
Parallelization on resampling within a stacked learner (ensemble/stack of classification learners) doesn't work
The below code works fine, however, I am interested to run it in parallel. I have tried different plans within future and future.apply but couldn't managed. Any help appreciated.
I am running on windows OS, 8…
Sheykhmousa
- 139
- 9
0
votes
0 answers
Non exportable references in future_map
I am experimenting for the first time with the furrr package, especially with the future_map group of functions.
I'm having trouble understanding how to avoid running into the problem of non-exportable references even with extremely basic functions.…
Violetta Zoffoli
- 15
- 6
0
votes
0 answers
Parallel Processing in R using Future package
I'm trying to run a bunch of tasks in parallel using 'Future' package and for some reason, it's not able to find the data frames that I want it to find. I've created the below sample program to show what I'm doing. Should I be exporting the Global…
0
votes
0 answers
Speed up processing of grouped data
Update:
I tried @Adam's approach, which did the trick. Using 8 cores, processing only took 20 minutes. This is great so far.
But I ran into another problem: foreach's %dopar% operator forks the workspace n times (equal to the number of cores…
Christian
- 707
- 1
- 7
- 15
0
votes
1 answer
How to apply mapping functions using furrr on tidygraph objects?
The parallel calculations on tidygraph centrality functions using furrr is throwing an error:
"Error in mutate_impl(.data, dots) : Evaluation error: This
function should not be called directly."
Here is my…
Geet
- 2,515
- 2
- 19
- 42
0
votes
1 answer
Parallelize multiple functions on the same object
Thanks to DAVID SCHOCH, I leveraged his function and slightly modified to create the following. It works perfectly fine.
Now, all I need is to calculate these variables i.e. run functions on the graph object H in parallel, maybe using purrr or…
Geet
- 2,515
- 2
- 19
- 42
0
votes
0 answers
R future, doSNOW, parallel and foreach for asynchronous execution
I'm struggling with choosing proper package for parallel asynchronous execution of scripts by external programs with R (R script also provides inputs and aggregates outputs), I know about four popular packages : future, doSNOW, parallel and foreach…
Qbik
- 5,885
- 14
- 62
- 93