Stack Overflow
Stack Overflow

Questions tagged [furrr]

This tag addresses the R package furrr intended to simplify the combination of purrr’s family of mapping functions.

This tag addresses the R package furrr intended to simplify the combination of purrr’s family of mapping functions.

68 questions
1
vote
1 answer

Tidyr::crossing more than 2 parameters (..1,..2,..3)

library(dplyr) library(fpp2) # for prison dataset library(hts) # forecasting function # prepare group time series prison.gts <- gts(prison/1e3, characters = c(3,1,9), gnames = c("State", "Gender", "Legal", …
Afiq Johari
  • 1,372
  • 1
  • 15
  • 28
1
vote
1 answer

What is furrr's "black magic?"

I use the R package furrr for most of my parallelization needs, and basically never have issues with exporting things from my global environment to the cluster. Today I did and I have no idea why. The package documentation seems to describe the…
Michael Roswell
  • 1,300
  • 12
  • 31
1
vote
0 answers

Mapping a keras NN over data lists in R

I am trying to figure out the correct method of going about applying a keras model to each of my lists. I have used the iris dataset and created 4 lists and the goal is to correctly predict versicolor or virginica (I omit setosa because I want a…
user113156
  • 6,761
  • 5
  • 35
  • 81
1
vote
1 answer

How can I correctly use the cluster plan in the R future (furrr) package

I am currently using furrr to create a more organized execution of my model. I use a data.frame to pass parameters to a function in a orderly way, and then using the furrr::future_map() to map a function across all the parameters. The function works…
topcat
  • 586
  • 1
  • 6
  • 30
1
vote
1 answer

Running parLapply and future_map inside another function unnecessarily copies large objects to each worker

I was looking for an alternative to furrr:future_map() because when this function is run inside another function it copies all objects defined inside that function to each worker regardless of whether those objects are explicitly passed…
Giovanni Colitti
  • 1,982
  • 11
  • 24
1
vote
1 answer

How can I configure future to download more files?

I have a lot of files I need to download. I am using download.file() function and furrr::map to download in parallel, with plan(strategy = "multicore"). Please advise how can I load more jobs for each future? Running on Ubuntu 18.04 with 8 cores. R…
SteveS
  • 3,789
  • 5
  • 30
  • 64
0
votes
0 answers

Running a series of adonis2() tests in parallel using a custom function + furrr

I've created a custom function that takes 4 inputs, runs some filtering, then performs a permanova using vegan::adonis2() The required inputs are: sample_data: a tibble primary_vars: name of a column in sample_data dists: distance object covars: a…
mestaki
  • 49
  • 4
0
votes
1 answer

How to increment a shared counter between parallel processes from the furrr package?

The example below parallelizes 64 Sys.sleep(1) instructions using the furrr package. The package progressr is used to display the progression of the parallel execution. This package allows to update some text details where I want to show the count…
pietrodito
  • 1,783
  • 15
  • 24
0
votes
0 answers

Distributing futures to workers with furrr using chunk_size

I'm sending jobs to 2 remote nodes using furrr. Each node is set to create 4 workers. I set this up using the following future::plan(): future::plan( list( future::tweak(future::cluster, workers = cl), future::tweak(future::multisession,…
Giovanni Colitti
  • 1,982
  • 11
  • 24
0
votes
0 answers

Multilevel parallelization on remote cluster of nodes using futures

I'm trying to distribute jobs across 4 linux worker nodes, each with 8 cores. I'm initiating the jobs from a Windows computer. Here is how I setup the cluster: worker_nodes <- c("node-1", "node-2") ssh_private_key_file <- "C:/user/.ssh/id_rsa" cl…
Giovanni Colitti
  • 1,982
  • 11
  • 24
0
votes
0 answers

Why is my function not recognising one particular argument when using future_map?

This code is designed to generate 1000 landscapes according to a poisson Matern cluster distribution algorithm, with the random allocation of points included to create dynamic landscapes. When running this code, the numerical object "radiusCluster"…
OpenSauce
  • 354
  • 2
  • 14
0
votes
0 answers

Improve performance API CAll in R with furrr

The code below performs an API request. Basically I inform two cities (origin and destination) and I must return which bus companies makes the connection between these cities. If there is not a line between these cities just ignores. Each city is…
L3A
  • 25
  • 4
0
votes
1 answer

How to use parallelization with raster::extract in R using furrr

I am unsure if this is a bug or a how-to. I posted this question here and was told to ask StackOverflow! library(tidyverse) library(tigris) library(elevatr) library(raster) library(sf) library(furrr) multco <- tigris::tracts(state = "OR", …
oatmilkyway
  • 429
  • 1
  • 6
  • 17
0
votes
0 answers

Issue with future and furrr, 1 processor only. R

Unfortunately, this is unlikely to be reproducible as I suspect it has to do with my setup. I'm on a windows machine, with a custom library location. Whenever I run future_map etc, only one core seems to be loaded. I suspect this is because my…
rwhit
  • 45
  • 6
0
votes
0 answers

parallel ocr with furrr::future_map

I would like to parallelize tesseract::orc using furr::future_map However, I get the error Detected a non-exportable reference (‘externalptr’ of class ‘tesseract’) which is related to non-exportable objects, I guess. I read that others also have…
ava
  • 840
  • 5
  • 19
1 2
3
4 5