Use this tag for questions specific to the future ecosystem in R - typically used for parallel and asynchronous processing. At the core of this ecosystem is Future API and the future package, which a larger set of packages then build upon.
Questions tagged [r-future]
94 questions
1
vote
1 answer
How to increase CPU usage in parallel processing in R
I am currently using the future package in R for some heavy parallel processing tasks.
When I examined the CPU usage while the script was running, I noticed that each parallel section is using only 2.3% of the CPU power on the machine (see below).…
johnlee14
- 65
- 4
1
vote
0 answers
Poor speed gain in using `future` for parallelization
I find that the speed gain in using the future (and furrr) package for parallelization in R is not satisfactory. In particular, the speed improvement is not close to linear. My machine has 4 workers, so I thought the speed gain should be around…
rick
- 63
- 6
1
vote
0 answers
R Shiny + Future + Plumber
Preamble
Inspired by the presentation by Barret Schloerke at studio::global(2021) (I will add the link as soon as it becomes available), I tried to implement an app to see the differences between using {future}, {plumber}, both or none into a Shiny…
Corrado
- 625
- 6
- 12
1
vote
0 answers
Assign variables to the global environment in a parallel loop
I am doing some heavy computations which I would like to speed up by performing it in a parallel loop. Moreover, I want the result of each calculation to be assigned to the global environment based on the name of the data currently processed:
fun <-…
koenniem
- 506
- 2
- 10
1
vote
1 answer
R load libraries only in the local scope
is it possible to have library() influence only the local scope?
For example, my script has a lapply loop that source() code in other scripts; these scripts load their own libraries, but I would like the namespace to be cleaned after the code in…
Bakaburg
- 3,165
- 4
- 32
- 64
1
vote
1 answer
How can I correctly use the cluster plan in the R future (furrr) package
I am currently using furrr to create a more organized execution of my model. I use a data.frame to pass parameters to a function in a orderly way, and then using the furrr::future_map() to map a function across all the parameters. The function works…
topcat
- 586
- 1
- 6
- 30
1
vote
1 answer
Running parLapply and future_map inside another function unnecessarily copies large objects to each worker
I was looking for an alternative to furrr:future_map() because when this function is run inside another function it copies all objects defined inside that function to each worker regardless of whether those objects are explicitly passed…
Giovanni Colitti
- 1,982
- 11
- 24
1
vote
1 answer
R package Future - why does a loop with remote workers hangs the local R session
Please let me know if you need an example, but I don't think it is necessary.
I've written a for loop that makes futures and store the results of each in a list. The plan is remote, say, made of 4 nodes on an internet machine.
After the 4th future…
Marciolegal
- 31
- 4
1
vote
1 answer
Is there a way to manually attach packages and globals with `future.apply::future_apply`
I am using R's excellent future package. And in the documentation it mentions %global% and %packages% for assigning global variables and packages to be evaluated in the future environment. But those seem to only work with %<-%.
My question is: is…
xiaodai
- 14,889
- 18
- 76
- 140
1
vote
1 answer
What is the idiomatic, flexible way to specify mutliprocess plan in future?
I just encountered the R packages furrr and future. I would love to use them to write flexible code that will use multiple cores if available on machines running Windows or OSX. I would love for the default # of "available" cores to be something…
Michael Roswell
- 1,300
- 12
- 31
1
vote
0 answers
downloadHandler uniquely ignores ongoing asynchronous process for the same user
I was experimenting promises and as I understand, as pointed out in this issue, an async process running for the same user is supposed to block execution of all other intra-session activities. However this doesn't seem to be a universal…
OganM
- 2,543
- 16
- 33
1
vote
1 answer
future waits for execution on single-core machine
I have 2 machines with identical versions of Ubuntu (16.04), R (3.4.4) and the future package (1.10.0). I am running the code below, that implements asyncronous programming
tic()
library(future)
plan(multisession)
mehAsync = function(){
…
OganM
- 2,543
- 16
- 33
1
vote
0 answers
R Google Compute Engine create cluster error
I'm trying to create an R cluster comprised on VMs on GCE. I've followed the example from googleComputeEngineR. I've pasted the code below.
######
vm_names <- paste0("cpu",1:2)
## specify the cheapest VMs that may get turned off
preemptible =…
CChen
- 23
- 6
0
votes
0 answers
sink function with future.callr
I have this R script:
test.R
library(future.callr)
plan(callr)
executeInternalActivity<-function(){
sink(file = "output.txt",append = TRUE)
print(Sys.time())
Sys.sleep(3)
print(Sys.time())
sink()
}
executeInternalActivity() …
Lev
- 693
- 1
- 8
- 24
0
votes
0 answers
Correct Implementation of Multiple Promises with Progress Update in R Shiny?
I am developing a Shiny application where I need to score a series of articles asynchronously by calling an external API using my own package. I aim to run multiple promises in parallel to enhance performance and wish to update a progress bar after…
Bakaburg
- 3,165
- 4
- 32
- 64