1

I have a few questions about the IPython parallel LoadBalancedView class which I couldn't find in the docs.

  1. How does a LoadBalancedView deal with nodes entering and exiting the pool? Does it recalculate loads and reassign? Otherwise once nodes join the older ones would still be backed up with tasks.
  2. When a task is added to the pool, is the task assigned immediately based on load or are tasks assigned just after a node "frees up" after completing its task? I'm just curious when the binding assignment happens, or if there is reshuffling all the time due to certain nodes being faster, etc.

Thanks!

lollercoaster
  • 15,969
  • 35
  • 115
  • 173

1 Answers1

0

I ran some tests. The LoadBalancedView assigns tasks to engines (nodes) immediately after being apply'd. If more nodes join later, the the load is not recalculated and the tasks are left to the original engines to complete. Only tasks created after the engine has registered with the controller can be assigned to that engine.

lollercoaster
  • 15,969
  • 35
  • 115
  • 173