2

I am iterating over a function that uses future_lapply(). I want to parallelize the outside lapply, for which I would use the following code:

plan(multisession, workers = 10)
future_lapply(X = 1:10, FUN = myfun)

Now, myfun() has its own future_lapply() inside. How do I configure the plan inside myfun() so that it does not mess up with the outside plan?

Inside myFun(), I am using

o.plan <- plan()
plan(sequential) 
on.exit(plan(o.plan), add = TRUE)

Is this the correct approach?

D1X
  • 5,025
  • 5
  • 21
  • 36
  • Could you add an argument to ```myfun(..., in_parallel = FALSE)``` and then change between ```future_lapply()``` and ```lapply()``` based on that arg? – Cole Nov 01 '20 at 11:39
  • @Cole yes, that is what I thought, but I was hoping for a better solution, without having to repeat code... – D1X Nov 01 '20 at 11:41
  • 1
    I'm curious about what a good solution to this is as well. In the meantime, you could do something like ```my_lapply = if (in_parallel) lapply else future_lapply``` and then just use ```my_lapply(...)``` – Cole Nov 01 '20 at 11:45
  • @Cole future_lapply has additional arguments like `future_seed`, this also complicates the call. – D1X Nov 02 '20 at 07:59
  • 1
    It may work to have a plan inside a plan, something like: ```plan(list(tweak(multisession, workers = 2)), tweak(multisession, workers = 8)))``` Beware of overloading things. – dca Mar 05 '22 at 04:38

0 Answers0