When using an ipyparallel cluster to process tasks in parallel, how do I iterate over the AsyncMapResult when some of the tasks have raised an exception?
All I get is the exception, but I can't figure out how to get at results of the tasks that have succeeded.
Here is some minimum sample code to demonstrate the issue:
import os
from ipyparallel import Client
# Create the ipyparallel client and LoadBalancedView
c = Client(os.path.expanduser('~/.ipython/profile_default/security/ipcontroller-client.json'))
lview = c.load_balanced_view()
# Define the function that will fail on x==True
def random_fail(x):
if x:
raise ValueError('haha')
return 'yay'
# Make a random sample of True/False
import random
tf = (True, False)
random_choices = [
random.choice(tf)
for x in xrange(100)
]
# Send the tasks to the ipyparallel cluster.
# amr is our AsyncMapResult object.
amr = lview.map(random_fail, random_choices, ordered=True)
amr.wait_interactive()
# At this point I only see the exceptions,
# but none of the 'yay' strings
for r in amr:
print r
# I could try indexing directly into the amr,
# but that still wouldn't work
for i in xrange(100):
if not random_choices[i]: # this should *not* cause random_fail() to fail
print amr[i]