6

I can't understand difference. Help me to watch this difference. And what about ProcessPoolExecutor, is his behavior the same?

def func(task):
    do_something(task)

tasks = [task for i in range(12)]
executor = ThreadPoolExecutor(4)
executor.map(func, tasks)
executor.shutdown(wait=True)  # ok, here the main thread waits for others

tasks = [task for i in range(12)]
executor = ThreadPoolExecutor(4)
executor.map(func, tasks)
executor.shutdown(wait=False)  # here it doesn't wait and what can happens bad?

tasks = [task for i in range(12)]
executor = ThreadPoolExecutor(4)
executor.map(func, tasks)  # if i don't call shutdown ?
kvdm.dev
  • 141
  • 1
  • 1
  • 12
  • 2
    https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.shutdown – njzk2 Feb 09 '15 at 19:19
  • 1
    I've read that. Is a method assigment just to wait all the threads to finish? – kvdm.dev Feb 09 '15 at 19:23
  • I don't understand your comment. – njzk2 Feb 09 '15 at 19:24
  • The `shutdown` method effectively just says "don't let this object accept any new tasks, and allow it to free its resources when the tasks already allocated are complete." The `wait` parameter just controls whether the method call blocks until the executor finishes "shutting down" or returns immediately while any running processes finish up (asynchronously). – Henry Keiter Feb 09 '15 at 19:28
  • i edited code. See pls, there are my final questions – kvdm.dev Feb 09 '15 at 19:43
  • Welcome to stackoverflow. This is not a nice edit because it kind of invalidates a big chunk of my answer. – Paulo Scardine Feb 09 '15 at 19:50
  • ok, thx just for a feedback and answers – kvdm.dev Feb 09 '15 at 19:54

1 Answers1

6

From the docs:

If wait is True then this method will not return until all the pending futures are done executing and the resources associated with the executor have been freed. If wait is False then this method will return immediately and the resources associated with the executor will be freed when all pending futures are done executing. Regardless of the value of wait, the entire Python program will not exit until all pending futures are done executing.

This covers the first two examples.

For the third one, since ThreadPoolExecutor adheres to the "context manager" protocol you can use it along the with statement in order to have the shutdown method automatically called for as soon as execution exits the with block.

The default is True if you omit the parameter - or if you use it as a context manager, so using it inside the with block is useless no matter the value of wait.

[edit]

i edited code. See pls, there are my final questions

You only call the shutdown method if you want to explicitly release all resources and ensure no new calls to submit or map will succeed. If you don't call shutdown (or use ThreadPoolExecutor as a context manager), resources may be released only when the entire Python program exits (and it will not exit until all pending futures are done).

Calling shutdown with wait==True or using ThreadPoolExecutor as a context manager will block until all pending futures are done executing.

The only use case I can think of for calling shutdown explicitly is:

executor = ThreadPoolExecutor(4)
try:
    executor.map(func, tasks)
finally:
    executor.shutdown(wait=False)

To give some context, this is the code snippet from the first version of this question:

def func(task):
    do_something(task)

tasks = [task for i in range(12)]
with ThreadPoolExecutor(4) as executor:
    executor.map(func, tasks)
    executor.shutdown(wait=True)  # what is happening here?

tasks = [task for i in range(12)]
with ThreadPoolExecutor(4) as executor:
    executor.map(func, tasks)
    executor.shutdown(wait=False)  # what is happening here?

tasks = [task for i in range(12)]
with ThreadPoolExecutor(4) as executor:
    executor.map(func, tasks)  # and without shutdown()?

Note that tasks = [task for i in range(12)] is redundant - you can use just executor.map(func, range(12)) as well.

Paulo Scardine
  • 73,447
  • 11
  • 124
  • 153
  • If the call to `shutdown` does not block when `wait=False`, can't you run some synchronous cleanup code while the threads are being cancelled in the background? If so, then it's *not* useless within a context manager scope. – Noah Jun 16 '22 at 19:46