Is it possible to execute code after pool.map finished its job in python?

Question

I want to collect time and failure messages in pool.map function calls and print them when all jobs are done. Is that possible in python?

Perhaps there is some event.. or smth else...

failures = []

def test():
   sleep(1)
   global failures
   failures.append('test message')

def main():
    data = [...]
    pool = ThreadPool(45)
    results = pool.map(test, data)
    pool.close()
    pool.join()

    # after all calls are finished i want to perform this
    print failures

The `map` function returns the result so if you simply return the error messages you are done, right? :) — Wolph, Sep 10 '14 at 20:20
Little bit confused after nodejs.. everything seems to by async! Will try to do that obvious thing now. — avasin, Sep 10 '14 at 20:22

score 2 · Accepted Answer · answered Sep 10 '14 at 20:24

Example (note that you can return anything, you could return [not raise] exceptions instead of normal results):

import random
import pprint
import multiprocessing

def test(value):
    if random.random() > 0.5:
        return 'Noes, we got an error for %d!' % value
    else:
        return 'All is ok for %d :)' % value

def main():
    data = range(10)

    pool = multiprocessing.Pool()
    results = pool.map(test, data)
    pool.close()
    pool.join()

    pprint.pprint(results)

if __name__ == '__main__':
    main()

Example output:

['All is ok for 0 :)',
 'Noes, we got an error for 1!',
 'Noes, we got an error for 2!',
 'Noes, we got an error for 3!',
 'All is ok for 4 :)',
 'Noes, we got an error for 5!',
 'All is ok for 6 :)',
 'Noes, we got an error for 7!',
 'All is ok for 8 :)',
 'Noes, we got an error for 9!']

Is it possible to execute code after pool.map finished its job in python?

1 Answers1