Stack Overflow
Stack Overflow

Questions tagged [furrr]

This tag addresses the R package furrr intended to simplify the combination of purrr’s family of mapping functions.

This tag addresses the R package furrr intended to simplify the combination of purrr’s family of mapping functions.

68 questions
0
votes
0 answers

Cannot replace purrr::map with furrr::future_map when using permuted data

I am trying to replace purrr::map with furrr:future_map to run linear regressions on a list of permuted dataframes but I get the following error: Error in as.data.frame.default(data) : cannot coerce class ‘"permutation"’ to a data.frame Here's…
boleneuro
  • 23
  • 4
0
votes
1 answer

Looping a function through a list of dataframes is very slow

I have a list, which contains 4438 dataframes with different sizes. I am not sure how to make a reproducible example, but the way I obtained the list is using the expand.grid function to have a dataframe with all the possible combination of…
Luong Nguyen Thanh
  • 11
  • 2
0
votes
2 answers

Efficient data.table rowwise and insertion of new columns

The dataset is very large and needs to be executed with parallelization. The following is a synthetic dataset: require(data.table) require(furrr) Names<-c("Estimate","Std.Error","t-value","Pr(>|t|)")…
user3647872
  • 85
  • 1
  • 9
0
votes
0 answers

How I can Speed up code built using purrr- Why is Furrr slower than purrr

I'd like to know if there is a way to speed up a code built with purrr package. I tried to convert it into furr and use the multisession option, but it is even slower than the sequential version. # rm(list =…
Anjeline
  • 21
  • 4
0
votes
1 answer

Error in future_map: argument ".f" is missing, with no default

Requesting your help or expert opinion on a parallelization issue I am facing. I regularly run an Xgboost classifier model on a rather large dataset (dim(train_data) = 357,401 x 281, dims after recipe prep() are 147,304 x 1159 ) for a multiclass…
landrower
  • 475
  • 6
  • 11
0
votes
1 answer

How to parallelize future_pmap() across multiple slurm nodes

I have access to a large computing cluster with many nodes each of which has >16 cores, running Slurm 20.11.3. I want to run a job in parallel using furrr::future_pmap(). I can parallelize across multiple cores on a single node but I have not been…
qdread
  • 3,389
  • 19
  • 36
0
votes
1 answer

Parallel GIS with `future`

I noticed that when loading furrr after raster, I am warned that values is masked: The following object is masked from ‘package:raster’: values I'm not 100% convinced that is the issue, but I am struggling to run raster::extract on sf…
Michael Roswell
  • 1,300
  • 12
  • 31
0
votes
1 answer

Is `Map()` when used in a `data.table` parallel? - R

From the data.table package website, given that: "many common operations are internally parallelized to use multiple CPU threads" I would like to know if that is the case when Map() is used within a data.table? The reason for asking is because I…
Buzz B
  • 75
  • 7
0
votes
1 answer

Using Purrr or Furrr to filter, and pass character vectors into additional functions

I have some very inefficient code that I am hoping someone can help me with. I don't have a great reprex, but have created a sample of the current code/workflow I'm using. Here's what I'm trying to do concisely Filter dataset into groups Pass…
Jazzmatazz
  • 615
  • 7
  • 18
0
votes
2 answers

How can I make this code run in parallel ? For loop

I am trying to run this simple for loop as a parallel process as it requires lots of compute power. Any thoughts? ##Load files and libraries--- library(tidyverse) library(caret) library(insight) library(MASS) library(mfx) library(furrr) for (i in…
Stat.Enthus
  • 335
  • 1
  • 12
0
votes
0 answers

r - downloading multiple files into separate subfolders with foreach - error with setwd

I have the following code which downloads a link into the appropriate folder > subfolder. This code works great although it is very slow. I have a couple of hundred .zip files that I am attempting to download so that they can be processed. Within…
iembry
  • 962
  • 1
  • 7
  • 23
0
votes
1 answer

R furrr plan returns errors when trying to tweak the multisession plan?

I am trying to set the plan I need but I am getting the following errors: no_cores <- availableCores() - 2 plan(multisession, workers = no_cores, lazy = T, gc = T) and the error is: Error in MultisessionFuture(expr = expr, envir = envir, substitute…
SteveS
  • 3,789
  • 5
  • 30
  • 64
0
votes
1 answer

Passing arguments to furrr::future_map using ellipsis (...)

I am trying to use furrr::future_pmap in R to replace purrr::pmap in a function call within another function. Presently I have it set up so pmap is passing other arguments using the ellipsis ... however when I try and do this using future_pmap I get…
Sarah
  • 3,022
  • 1
  • 19
  • 40
0
votes
0 answers

Run an R function in parallel / using furr / Syntax

I have this call on igraph object (gr_muc) that changes the weights of the vertices. E(gr_muc)[edges_from]$weight <- E(gr_muc)[edges_from]$weight * 2 E(gr_muc)[edges_to]$weight <- E(gr_muc)[edges_to]$weight * 2 Is it possible to use furrr or some…
Andreas
  • 397
  • 4
  • 18
  • 37
0
votes
2 answers

Environment/scoping in furrr: nesting get() in future_map()

I'm interested in learning more about how furrr finds stuff from the global environment, and asked generally about the black magic it performs. Here's a specific example of a behavior I didn't understand and could use some help with: What do I need…
Michael Roswell
  • 1,300
  • 12
  • 31
1 2 3
4
5