Given the following code:
ids = [[1,2,3], [4,5], [6,7,8,9]]
it = [('a','b', 'c', ids[i]) for i in range(len(ids))]
# [('a', 'b', 'c', [1, 2, 3]), ('a', 'b', 'c', [4, 5]), ('a', 'b', 'c', [6, 7, 8, 9])]
p = multiprocessing.Pool(2)
j = p.starmap(f, it)
p.close()
p.join()
and function of form f(str, str, str, list)
Is this an approach with the least overhead? (time and space?)
'a', 'b' and 'c' come from configuration
Since the first 3 arguments are always the same, you might do a bit better with:
from functools import partial
import multiprocessing
ids = [[1,2,3], [4,5], [6,7,8,9]]
p = multiprocessing.Pool(2)
results = p.map(partial(f, 'a', 'b', 'c'), ids)
This saves having to create the it list. It is certainly neater. At any rate, it would have been more Pythonic to create it as:
it = [('a', 'b', 'c', id) for id in ids]
