3

I'm using parallel::mclapply() with R 4.0.1 and getting the following warning: "scheduled cores ... did not deliver results, all values of the jobs will be affected".

Here the result of my investigation: inspecting the function source code, I realized that it happens when the vector dr is not all TRUE. This means that for some cores the second condition inside the for loop below (is.raw(a)) is never executed. a is the value returned by readChild(), that if returned raw data at least once, the condition would be verified at least once. So I'm thinking that readChild() is returning NULL.

readChild and readChildren return a raw vector with a "pid" attribute if data were available, an integer vector of length one with the process ID if a child terminated or NULL if the child no longer exists (no children at all for readChildren).

I ask you to validate or reject my conclusions. Finally if true what are the possible reasons?

    while (!all(fin)) {
        s <- selectChildren(ac[!fin], -1)
        if (is.null(s)) break # no children -> no hope we get anything (should not happen)
        if (is.integer(s))
            for (ch in s) {
                a <- readChild(ch)
                if (is.integer(a)) {
                    core <- which(cp == a)
                    fin[core] <- TRUE
                } else if (is.raw(a)) {
                    core <- which(cp == attr(a, "pid"))
                    job.res[[core]] <- ijr <- unserialize(a)
                    if (inherits(ijr, "try-error"))
                        has.errors <- c(has.errors, core)
                    dr[core] <- TRUE
                } else if (is.null(a)) {
                    # the child no longer exists (should not happen)
                    core <- which(cp == ch)
                    fin[core] <- TRUE
                }
            }
    }
Lstat
  • 1,450
  • 1
  • 12
  • 18
lulions
  • 448
  • 4
  • 10
  • Welcome to SO! Would it be possible to include a minimal reproducible example? See the https://stackoverflow.com/help/minimal-reproducible-example for guidance. – Lstat Jul 09 '20 at 09:54
  • Thank you, I'm trying to reproduce the warning message. At the moment I got it only in AWS Sagemaker with a ml.m5.12xlarge instance and a very large amount of data. – lulions Jul 09 '20 at 13:23

1 Answers1

4

This error message can occur when the child process dies/crashes, e.g.

> y <- parallel::mclapply(1:2, FUN = function(x) if (x == 1) quit("no") else x)
Warning message:
In parallel::mclapply(1:2, FUN = function(x) if (x == 1) quit("no") else x) :
  scheduled core 1 did not deliver a result, all values of the job will be affected

> str(y)
List of 2
 $ : NULL
 $ : int 2

That a child process completely dies is of course not good. It can happen for several reasons. My best guess is that you parallelize something that must not be parallelized. Forked processing (=mclapply()) is known to be unstable with code that multi-thread, among other things.

For what's it worth, if you use the future framework instead (disclaimer: I'm the author), you'll get an error message that is a bit more informative, e.g.

> library(future.apply)
> plan(multicore)

> y <- future_lapply(1:2, FUN = function(x) if (x == 1) quit("no") else x)
Error: Failed to retrieve the result of MulticoreFuture (future_lapply-1) from
the forked worker (on localhost; PID 19959). Post-mortem diagnostic: No process
exists with this PID, i.e. the forked localhost worker is no longer alive.
HenrikB
  • 6,132
  • 31
  • 34
  • Is there a way to get a backtrace for either future or mclapply? In either message, it still doesn't tell me, the user, where the code is breaking. – James Dalgleish Apr 13 '21 at 20:20
  • I don't think it would ever be possible to get the call stack after such a harsh event as the worker crashing. It's like calling `tools::pskill(Sys.getpid())` in your main R session and then trying to get a traceback. – HenrikB Jan 27 '22 at 18:31