Use this tag for questions specific to the future ecosystem in R - typically used for parallel and asynchronous processing. At the core of this ecosystem is Future API and the future package, which a larger set of packages then build upon.
Questions tagged [r-future]
94 questions
3
votes
1 answer
Asynchronous programming in R
Overview
I am writing a program (in R) that makes API calls at certain designated times. The API calls take a while, but I need the timer (main loop) to continue counting while the API call is made. To do so, I need to "outsource" the API call to…
Brigadeiro
- 2,649
- 13
- 30
3
votes
1 answer
How to set .libPaths (checkpoint) on workers when running parallel computation in R
I use the checkpoint package for reproducible data analysis. Some of the computations take a long time to compute, so I want to run those in parallel.
When run in parallel however the checkpoint is not set on the workers, so I get an error message…
needRhelp
- 2,948
- 2
- 24
- 48
3
votes
2 answers
Why do my second-level futures execute serially?
I am trying to replicate the examples in the topology vignette of the future package. To quote:
Futures can be nested in R such that one future creates another set of futures
and so on. This may, for instance, occur within nested for loops…
akraf
- 2,965
- 20
- 44
3
votes
1 answer
How to do parallel computing inside a cluster with the R future package?
I would like to distribute jobs (with a for loop) inside nodes of a cluster (several machines). I try to use the R package future to do that. I didn't know if it's the best way to do that; I tried to use foreach of the doParallel package, but I…
Axel971
- 43
- 4
2
votes
2 answers
How to convert foreach into a function?
I am using a foreach to calculate the correlation coefficients and p values, using the mtcars as an example ( foreach is overkill here but the dataframe I'm using has 450 obs for 3400 variables). I use combn to get rid of duplicate correlations and…
pemb_bex6789
- 83
- 6
2
votes
1 answer
Ensure reproducibility across `purrr::map()` and `furrr::future_map()`
I am running simulations where some computing should be parallelized and some should not.
I am trying to figure out how to ensure reproducibility across purrr::map() and furrr::future_map() so that they yield the same result.
For some reason, I…
Dan Chaltiel
- 7,811
- 5
- 47
- 92
2
votes
1 answer
Check if multisession is running in future R
Is there a quick way to check if multiple processes were generated by future package on the current R?
In other words, to check whether plan(multisession) or similar was run in R?
Matthew Son
- 1,109
- 8
- 27
2
votes
1 answer
How can you use selecticizeGroupInput with a future_promise?
I am using a future promise to call my data more efficiently. However, I'm finding that my filters (called through selecticizeGroupServer) do not work with the future promise.
See attached for a minimal reproducible example. The filters work as…
jstuebner
- 33
- 4
2
votes
0 answers
Output to console in run-time from future_map
I have following code which is supposed to run in parallel with furrr.
plan(multisession)
1:10 %>% furrr::future_map(function(x) {
print("Start")
Sys.sleep(1)
print("End")
x
}, .progress = TRUE)
By specifying .progress = TRUE I can…
vkolesnik
- 367
- 2
- 13
2
votes
1 answer
I get an error when using {future} and {furrr} functions within a Golem Shiny App, what does it come from?
I am currently working on a Golem Shiny App called "package_name" (which is a requirement for me) for which some functions I created need to use functions from the {furrr} and {future} packages.
However, whenever I try to run them, I get the…
Henri Freixe
- 33
- 3
2
votes
0 answers
How to split dataframe for future_map for optimal performance
I am running a data preparation script which involves a step in which the rows of the dataframe are split up into multiple observations. This step takes quite some time and I'm trying to optimize the use of future to get some improvements. The…
arohland
- 96
- 6
2
votes
0 answers
How can I print messages to console when using `future_lapply()`?
I am running a function in parallel in R using future_lapply(), but I want to print a message to the R console.
I have something like this:
library(future.apply)
plan(multisession, workers = 2)
future_lapply(1:12, function(idx) {
Sys.sleep(5)
…
D1X
- 5,025
- 5
- 21
- 36
2
votes
0 answers
using future_lapply() inside future_lapply()
I am iterating over a function that uses future_lapply(). I want to parallelize the outside lapply, for which I would use the following code:
plan(multisession, workers = 10)
future_lapply(X = 1:10, FUN = myfun)
Now, myfun() has its own…
D1X
- 5,025
- 5
- 21
- 36
2
votes
2 answers
R furrr: authenticate an API on each future process before running the computation
I am running a parallel computation using furrr in R. The computation require access to a web API and an authentication needs to take place. If I run a parallel process, each process needs to authenticate.
In the below, I have 6 processes. So I…
Courvoisier
- 904
- 12
- 26
2
votes
1 answer
R package future: Initialization of plan() failed
Trying this code
library(future)
library(foreach)
ncores <- 3
cl <- parallel::makeCluster(3)
avail <- bigstatsr::FBM(ncores, 1, type = "integer", init = 1)
doFuture::registerDoFuture()
res <- vector("list", 5)
for (i in seq_along(res)) {
while…
F. Privé
- 11,423
- 2
- 27
- 78