Wondering if you could have multiple functions in a task for process pool python

Question

def task():
    print("Executing our Task on Process {}".format(os.getpid()))

def main():
    executor = ProcessPoolExecutor(max_workers=3)
    task1 = executor.submit(task)
    task2 = executor.submit(task)

if __name__ == '__main__':
    main()

So i saw this program from this website : https://tutorialedge.net/python/concurrency/python-processpoolexecutor-tutorial/

And i was just wondering if you can have multiple functions under task. Like is process pool only supposed to do one task? Because i have a program that has a function for every task, and wondering how i could convert it to a process pool executor.

Any help would be appreciated. Thank you

score 1 · Answer 1 · answered Apr 16 '20 at 04:03

A ProcessPoolExecutor is a subclass of Executor. The Executor.submit() method looks like this:

 submit(fn, *args, **kwargs)

Which according to the documentation:

Schedules the callable, fn, to be executed as fn(*args **kwargs) and returns a Future object representing the execution of the callable.

So it should be able to run any callable - there is no relation between the executor & the kind of tasks it can execute.

Here is a simple extension to the code you provided that you can use to try it out:

def task_one():
    print("Executing our first Task on Process {}".format(os.getpid()))

def task_two():
    print("Executing our second Task on Process {}".format(os.getpid()))

def main():
    executor = ProcessPoolExecutor(max_workers=3)
    executor.submit(task_one).result()
    executor.submit(task_two).result()

if __name__ == '__main__':
    main()

Wondering if you could have multiple functions in a task for process pool python

1 Answers1