Parallel version of sapply

Question

Do we have parallel version of sapply, as we have mclapply in parallel package which is a version of lapply.

lapply and sapply is pretty much the same function except that sapply simplifies the results, so what sapply can do lapply can do as well, you just will need to do some postprocessing, ie `do.call("cbind", df)` or `do.call("c", df)` — grrgrrbla, Jun 25 '15 at 12:48
You're looking like a dupe of http://stackoverflow.com/questions/14759905/data-table-and-parallel-computing (at least insomuch as you've tagged data.table; there's surely a general R parallelization question, too) — Frank, Jun 25 '15 at 12:54

score 9 · Answer 1 · answered Nov 22 '19 at 19:37

As @RHertel points out, the snow package has apply family options, but it does not count with an mcsapply function as it is asked. If you take a look a the implementation of sapply, it is simply a call to lapply with some more post processing:

sapply
#> function (X, FUN, ..., simplify = TRUE, USE.NAMES = TRUE) 
#> {
#>     FUN <- match.fun(FUN)
#>     answer <- lapply(X = X, FUN = FUN, ...)
#>     if (USE.NAMES && is.character(X) && is.null(names(answer))) 
#>         names(answer) <- X
#>     if (!isFALSE(simplify) && length(answer)) 
#>         simplify2array(answer, higher = (simplify == "array"))
#>     else answer
#> }
#> <bytecode: 0x559a53feef18>
#> <environment: namespace:base>

^{Created on 2019-11-22 by the reprex package (v0.3.0)}

Since the mclapply function is equivalent to lapply, you could actually write your own replacing some parts of sapply, here is an example:

# An mc-version of the sapply function.
mcsapply <- function (X, FUN, ..., simplify = TRUE, USE.NAMES = TRUE) {
  FUN <- match.fun(FUN)
  answer <- parallel::mclapply(X = X, FUN = FUN, ...)
  if (USE.NAMES && is.character(X) && is.null(names(answer))) 
    names(answer) <- X
  if (!isFALSE(simplify) && length(answer)) 
    simplify2array(answer, higher = (simplify == "array"))
  else answer
}

# Testing it out
library(parallel)

ans0 <- mcsapply(1:20, function(i) rnorm(1e4), mc.cores = 4)
ans1 <- sapply(1:20, function(i) rnorm(1e4))

# Same structure
str(ans0)
#>  num [1:10000, 1:20] 0.1792 0.1581 -0.1293 -0.0324 0.1836 ...
str(ans1)
#>  num [1:10000, 1:20] 1.304 1.355 -1.387 1.07 0.582 ...

^{Created on 2019-11-22 by the reprex package (v0.3.0)}

Finally, you can always create a Fork cluster and use parSapply instead.

score 0 · Answer 2 · answered Jun 25 '15 at 12:50

0

The snow package provides parallel versions of apply() and related functions.

answered Jun 25 '15 at 12:50

RHertel

23,412
5
38
64

happyspace · Answer 3 · 2019-11-24T23:59:08.910

-1

In the event that your output could be coerced into an atomic vector, you can use unlist(mclapply(....)).

edited Nov 24 '19 at 23:59

answered Nov 13 '19 at 02:14

happyspace

113
1
2
12

This is not necessarily true since you are assuming that the output of `mclapply` can be coerced to an atomic vector, which is not always the case. – gvegayon Nov 22 '19 at 19:37

Parallel version of sapply

3 Answers3