Multisession parallelism with a shared stderr redirect

Question

I am trying to run R code with multisession parallelism such that all the error messages redirect to the same file. However, the sink() cannot be created.

library(parallel)
cl <- makePSOCKcluster(2)
f <- function(){
  withr::with_message_sink("messages.txt", Sys.sleep(10))
}
clusterCall(cl = cl, fun = f)

## Error in checkForRemoteErrors(lapply(cl, recvResult)) :
##   2 nodes produced errors; first error: Cannot establish message sink when another sink is active.
## Calls: clusterCall -> checkForRemoteErrors
## Execution halted

---

Edit

Given some of the responses, I should elaborate on the purpose of this post. I am developing drake, an R package with multiple parallel backends. Today, I implemented a new hook argument to make(), which just wraps individual parallel jobs in a function of the user's choice. What I am really looking for is a hook that silences the console regardless of parallel backend. Backends in the current development version include

• parallel::mclapply()
• parallel::parLapply()
• base::lapply() (via parLapply() for one job)
• make -j with a proper Makefile
• future::sequential
• future::multicore
• future::multisession
• future.batchtools backends listed here

I thought I found a hook that worked for stderr.

hook <- function(){
  withr::with_message_sink("messages.txt", Sys.sleep(10))
}

However, withr::with_message_sink() does not let me sink multiple workers to the same file for the parLapply() or future::multisession backends.

Answer 1

Can you just use sink?:

library(parallel)
cl <- makePSOCKcluster(2)
clusterApply(cl, seq_along(cl), function(i) workerID <<- i)


f <- function(){
  outtxt <- paste(workerID, "messages.txt", sep="_")
  print(outtxt)
  sink(outtxt)
  Sys.sleep(10)
  sink()
}
clusterCall(cl = cl, fun = f)

stopCluster(cl)