Stack Overflow
Stack Overflow

Questions tagged [pmap]

161 questions
6
votes
1 answer

Julia: Using pmap correctly

Why doesn't this do what I think it should: benjamin@benjamin-VirtualBox:~$ julia -p 3 julia> @everywhere(function foom(bar::Vector{Any}, k::Integer) println(repeat(bar[2],bar[1])); return bar; end) julia> foo={{1,"a"},{2,"b"},{3,"c"}} julia>…
bdeonovic
  • 4,130
  • 7
  • 40
  • 70
5
votes
1 answer

furrr / purrr progressr progress bar not at all synched with the progress of the computation

I want to create a function that takes a function and applies it once for every row in a tibble with arguments stored in the correspondingly named columns of the tibble I realize that this sounds a bit odd, but I want the user facing function /…
Fredrik Karlsson
  • 485
  • 8
  • 21
5
votes
1 answer

purrr::pmap with other default inputs

I am wondering how to use pmap() function if I have more than 3 inputs as parameters to map into a function with other default inputs. Here is a reproducible example: a=c(5, 100, 900) b=c(1, 2, 3) ablist=list(mean=a,sd=b) pmap(ablist, ~rnorm(…
Joanna
  • 663
  • 7
  • 21
5
votes
1 answer

Why is pmap|reducers/map not using all cpu cores?

I'm trying to parse a file with a million lines, each line is a json string with some information about a book (author, contents etc). I'm using iota to load the file, as my program throws an OutOfMemoryError if I try to use slurp. I'm also using…
eugecm
  • 1,189
  • 8
  • 14
5
votes
1 answer

pmap slow for toy example

I'm testing out parallelism in Julia to see if there's a speedup on my machine (I'm picking a language to implement new algorithms with). I didn't want to dedicate a ton of time writing a huge example so I did the following test on release version…
Helix
  • 171
  • 6
5
votes
1 answer

binding & pmap interaction change?

There are several somewhat old blog posts out there advising caution when mixing dynamic variables, binding, and pmap, e.g. here, where we get the following code snippet: user=> (def *foo* 5) #'user/*foo* user=> (defn adder [param] …
ben w
  • 2,490
  • 14
  • 19
4
votes
3 answers

Assign columns as arguments to purrr::pmap by using their names instead of position

I'm trying to use the pmap function from the purrr package to loop over different columns of a tibble. Each column contains certain model parameters (e.g. sample size, coefficients, etc.) that I would like to use as input for my function. Each row…
Rasul89
  • 588
  • 2
  • 5
  • 14
4
votes
1 answer

purrr pmap function arguments

Functional programming with either *apply or purrr is my bread and butter yet I do not understand how pmap handles function arguments. I can use their corresponding variable names (given a data frame as input) and do not have to worry about the…
mnist
  • 6,571
  • 1
  • 18
  • 41
4
votes
0 answers

Why pmap shows a mapping that strace doesn't?

I am investigating memory usage of a process that (unexpectedly) reserves terabytes of VIRT. If I do pmap for that process, I can see some beefy allocation, e.g.: 00007f39ea671000 317288696K rw--- [ anon ] I am running my process under strace…
pepyakin
  • 2,217
  • 19
  • 31
4
votes
1 answer

purrr pmap in list-columns data frame

When I have a list-column data frame (df1) with two input columns (a and b), I can obtain a new column using map2 d1 <- mtcars %>% slice(1:10) %>% group_by(cyl) %>% nest(.key = "a") d2 <- mtcars %>% slice(11:20) %>% group_by(cyl) %>% nest(.key =…
danilinares
  • 1,172
  • 1
  • 9
  • 28
4
votes
2 answers

clojure pmap - why aren't i using all the cores?

I'm attempting to use the clojure pantomime library to extract/ocr text from a large number of tif documents (among others). My plan has been to use pmap for to apply the mapping over a sequence of input data (from a postgres database) and then…
joefromct
  • 1,506
  • 13
  • 33
4
votes
1 answer

Implementing pmap from scratch. Why my implementation slow?

I'm new to Erlang, so for training I try to implement standard functions from scratch. I've tried to create parallel implementation of map/2 function from lists module. But my implementation works very slow. Could you point me, if I did any…
stemm
  • 5,960
  • 2
  • 34
  • 64
3
votes
1 answer

Why do I see big differences in memory usage with pmap for the same process on 32bit and 64bit Linux?

I was setting up a new server (64bit Debian) and, in an attempt to make the apache process as small as possible, disabled any modules that I didn't need. I then compared the pmap output to apache on a 32bit Debian box with lots more modules turned…
Lemo
  • 211
  • 3
  • 9
3
votes
3 answers

Transform a list of lists into a differently arranged list of lists to use for pmap

I've been struggling really hard to get pmap() to work and misunderstood the structure of the list that I am supposed to feed it. Basically, I ended up with a list of lists, before, but it's not in the correct arrangement for pmap. Here's a minimum…
Angelo
  • 2,936
  • 5
  • 29
  • 44
3
votes
1 answer

Have a function accept a list column of parameters

Related to Pass a list of lists of unquoted character parameters to an apply/map/pmap call. I have a function that looks like this: causal_med_so <- function(predictor, mediator, outcome, data, ...){ predictor <- rlang::ensym(predictor) mediator…
Raoul Duke
  • 435
  • 3
  • 13
1
2
3
…
10 11