Can't use namedtuple with concurrent.futures?

Question

The following code runs fine:

import itertools
import concurrent.futures

test_tuple = (1, "test")

iterations = range(1, 100)


def print_func(print_int, test_tuple):
    print(print_int, test_tuple)


def parallel_func():
    with concurrent.futures.ProcessPoolExecutor() as executor:
        executor.map(print_func, iterations, itertools.repeat(test_tuple))


if __name__ == "__main__":
    parallel_func()

However if I use a namedtuple:

import itertools
import concurrent.futures
from collections import namedtuple

TestNamedTuple = namedtuple("test", "id, name")
test_tuple = TestNamedTuple(1, "test")

iterations = range(1, 100)


def print_func(print_int, test_tuple):
    print(print_int, test_tuple.id, test_tuple.name)


def parallel_func():
    with concurrent.futures.ProcessPoolExecutor() as executor:
        executor.map(print_func, iterations, itertools.repeat(test_tuple))


if __name__ == "__main__":
    parallel_func()

The terminal locks up with no error message to be seen. Any ideas?

Weird problem, at first sight I'm sure the `itertools.repeat(test_tuple)` argument is passed as the `timeout` argument to `executor.map`, so it's already a mistery why it works at all :D — Szabolcs, Nov 19 '20 at 14:41
Does this answer your question? [Using collections.namedtuple with ProcessPoolExecutor gets stuck in a few cases](https://stackoverflow.com/questions/63609985/using-collections-namedtuple-with-processpoolexecutor-gets-stuck-in-a-few-cases). If you were to switch to useing `multiprocessing.Pool`, you will get a pickling error, i.e. the `namedtuple` cannot be pickled. — Booboo, Nov 19 '20 at 14:59

Can't use namedtuple with concurrent.futures?

0 Answers0