parLapply: unused argument (MoreArgs

Question

I was trying to debug a parLapply function that I am working on when I encountered an unexpected problem: the MoreArgs argument to parLapply is not working! It's 5AM at the moment, and I must be fatigued and missing something really obvious and silly, but I cannot for the life of me see what it is. Anyway, here is the minimal example that is giving me trouble:

require(parallel)

cluster <- makeCluster(3)
testFUN <- function(n=10, min=0, max=100, outname="test1.Rdata"){
    nums <- runif(n, min=min, max=max)
    total <- sum(nums)
    save(total, file = outname)
    return(total)

}


#just to make sure the function works
testFUN(n=5, min=4, max=150, outname = "test0.Rdata")

#make sure the cluster can get the functions
clusterExport(cluster, list("testFUN", "runif", "sum", "save"))
nlist <- list(first=5, second=10, third=15)
parLapply(cl = cluster, nlist, testFUN, outname=c("test1.Rdata", "test2.Rdata", "test3.Rdata"), MoreArgs=list(min=33, max=110))

stopCluster(cluster)

and the output was:

Error in checkForRemoteErrors(val) : 
  3 nodes produced errors; first error: unused argument (MoreArgs = list(min = 33, max = 110))

I think `MoreArgs` is used in `Map`. For `parLapply` just passing `min = 33, max = 110` directly inside `parLapply` should work. It looks like you need `clusterMap` though (over nlist and outname) — konvas, Aug 03 '18 at 09:28
Well, that changes to a different error which is actually even more confusing to me: `> parLapply(cl = cluster, nlist, testFUN, outname=c("test1.Rdata", "test2.Rdata", "test3.Rdata"), min=33, max=110) Error in checkForRemoteErrors(val) : 3 nodes produced errors; first error: invalid 'description' argument` — Stonecraft, Aug 03 '18 at 09:34
You're passing two strings to `save(..., file = outname)`. Sleep on and look at it when you wake you with fresh eyes and a fresh mind. — HenrikB, Aug 03 '18 at 10:09

fdetsch · Answer 1 · 2022-07-28T07:13:39.160

For future reference, here is a workable example using clusterMap() (as already indicated by @konvas), which is easier to use in this case since there are two control variables. Also, there is no need for clusterExport() before the loop as long as there is no anonymous function involved that needs to access objects from the global environment.

## open cluster and define function, then..

nlist = list(
  first = 5
  , second = 10
  , third = 15
)

outname = sprintf(
  "test%s.Rdata"
  , 1:3
)

clusterMap(
  cl = cluster
  , fun = testFUN
  , nlist
  , outname
  , MoreArgs = list(
    min = 33
    , max = 110
  )
)
# $first
# [1] 335.4677
# 
# $second
# [1] 681.2613
# 
# $third
# [1] 1065.619

stopCluster(
  cluster
)

parLapply: unused argument (MoreArgs

1 Answers1