Every time I run very time consuming task with iPython parallel (ipyparallel), I receive a popup message "Kernel Restarting The kernel appears to have died. It will restart automatically."
What is a good way for debugging this situation? I print ever lines on my program, and probably right after finishing all process finish the kernel die and restarting.
i.e., in the following code, after executing run_parallel for several times (usually 10 times), the kernel dies before printing "finish parallel" message. Then, before dying, "finish run 99" has been printed every time.
Therefore, I guess the kernel dies when returning from parallel engines after finishing all parallel process. But I can not how to debug and fix it.
from IPython.parallel import Client
from IPython import parallel
from IPython.parallel import require
def run(x):
# run process once for x
...
print "finish run %d" % x
return result
def run_parallel(num):
print("run_parallel: %d times" % num)
rc = parallel.Client()
dview = rc[:]
try:
res = dview.map_async(run, range(100))
wait_watching_stdout() # Real-time output from engines in IPython parallel (http://stackoverflow.com/questions/18086299/real-time-output-from-engines-in-ipython-parallel)
print ("finish parallel")
result = res.get()
finally:
rc.close()
return result
all_result = []
for num in range(500):
r = run_parallel(num)
all_result.append(r)
My Environment
- Ubuntu 14.04
- python 2.7.10 on anaconda 2.4.0
- iPython 4.0.0
- ipyparallel 5.0.1
