2

IPython.parallel's load balanced interface has been great. I used it to run complicated scientific computations on a small cluster environment. Several houndred problems, each taking 5~10 minutes to complete on a PC, were sent to the cluster. I had no problem getting the results back until I started submitting more strenuous tasks to the cluster. Some engines were unregistered even though the task assigned completed successfully. I use "ipcontroller --ping=120000" to circumvent the problem, but IPython now tells me "Task farming is disabled". What is causing the problem and what should I do?

By the way, I use ssh mode on the cluster.

dAvid
  • 21
  • 4
  • Have you used `c.TaskSchedule.scheme_name = 'pure'` in your `ipcontroller_config.py`? – minrk Nov 20 '15 at 19:45
  • @minrk: I wonder if this would disable my ability to run tasks on selected engines. I have both multi-threaded and single-threaded tasks. All engines are started at the beginning. A subset of engines, one from each physical node, is selected and used when running the multi-threaded task. – dAvid Jan 10 '16 at 13:13
  • @minrk: Also, sorry for the late reply. I was busy amending other parts of my project. – dAvid Jan 10 '16 at 13:16

0 Answers0