How can I catch when a thread dies in ThreadPoolExecutor()?

Question

I have some very simple python code that runs a bunch of inputs through various processes via ThreadPoolExecutor(). Now, sometimes one or more of the threads dies quietly. It is actually great that the rest of the threads continue on and the code completes, but I would like to put together some type of summary that tells me which, if any, of the threads have died.

I've found several examples where folks want the whole thing to shut down, but haven't seen anything yet where the process continues on and the threads that have hit errors are just reported on after the fact.

Any/all thoughts greatly appreciated!

Thanks!

import concurrent.futures as cf

with cf.ThreadPoolExecutor() as executor:
     executor.map(process_a, process_a_inputs)
     executor.map(process_b, process_b_inputs)

When you say a thread dies quietly - do you mean a concurrent task fails? Does the future contain a result or an exception? — Useless, Apr 09 '20 at 13:49
From the docs of ``Executor.map``: ["If a func call raises an exception, then that exception will be raised *when its value is retrieved from the iterator*."](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.map) Basically, the code is purposely throwing away all exceptions by ignoring the results of ``.map``. Are you fine with receiving just the first exception, or do you expect to observe multiple exceptions? If the latter, do these need to actually propagate or just be logged and discarded, e.g. via ``print``? — MisterMiyagi, Apr 09 '20 at 13:57
You could provide a `ThreadFactory` when you create the `ThreadPoolExecutor`. Every time your factory is asked to create a new thread (and maybe at other times of your choosing) your factory could check up on the status of the threads that it previously created. You can count the thread deaths that way, but you won't be able to distinguish threads that died because of an exception thrown by a task from threads that died when the executor decided to shut them down because it had too many threads for the current work load. — Solomon Slow, Apr 09 '20 at 16:13
@Useless Correct, when a concurrent task fails. Based on your and other's responses I started playing around with the future. When I use the future, it does catch the exception. However, it looks like the failing concurrent task interrupts subsequent tasks, which I don't want. — BMcG, Apr 09 '20 at 16:46
@MisterMiyagi I would expect to observe multiple exceptions. I'm basically trying to construct an email saying "process_a failed with ['123', 'abc', 'xyz'] inputs. So that all the other concurrent tasks are unaffected, and I can circle back on why '123', 'abc', and 'xyz' didn't work. — BMcG, Apr 09 '20 at 16:49
@SolomonSlow Thanks much for your response! Unfortunately I do hope to understand which thread died (ie what inputs). — BMcG, Apr 09 '20 at 16:50
@BMcG, There won't be any way for you to identify the last task that a `ThreadPoolExecutor` pool thread executed after the thread has died. IMO, you should quit trying to game the library, and just write your own thread pool that behaves the way you want it to behave. It's not all that difficult. — Solomon Slow, Apr 09 '20 at 17:08

score 2 · Accepted Answer · answered Apr 09 '20 at 18:50

Executor.map does not support gathering more than one exception. However, its code can easily be adapted to return the arguments on which a failure occurred.

def attempt(executor: 'Executor', fn: 'Callable', *iterables):
    """Attempt to ``map(fn, *iterables)`` and return the args that caused a failure"""
    future_args = [(self.submit(fn, *args), args) for args in zip(*iterables)]

    def failure_iterator():
        future_args.reverse()
        while future_args:
            future, args = future_args.pop()
            try:
                future.result()
            except BaseException:
                del future
                yield args
    return failure_iterator()

This can be used to concurrently "map" arguments to functions, and later retrieve any failures.

import concurrent.futures as cf

with cf.ThreadPoolExecutor() as executor:
     a_failures = attempt(executor, process_a, process_a_inputs)
     b_failures = attempt(executor, process_b, process_b_inputs)
     for args in a_tries:
         print(f'failed to map {args} onto a')
     for args in b_tries:
         print(f'failed to map {args} onto b')

You mean, if there is some issue in a thread we can return it as error response. And deal with it during collection of results ? — Arnab Mukherjee, Jul 20 '21 at 05:40

How can I catch when a thread dies in ThreadPoolExecutor()?

1 Answers1

Linked