2

When submitting task graphs using fire_and_forget, is it possible to later (in a new process/client) cancel those tasks (e.g. by key name)? And will that also cancel all dependent tasks, or are these also 'fire_and_forget'-like?

Vincent Schut
  • 141
  • 5

1 Answers1

2

Yes, you can create a new future with the keyname

from dask.distributed import Future

future = Future(key_name, my_client)
future.cancel()

Forcing cancellation even in the face of multiple clients seems reasonable (fire-and-forget is considered its own client). Implemented here: https://github.com/dask/distributed/pull/1408 . In version > 1.18.3 you will be able to use the force=True keyword

future.cancel(force=True)

This will cancel the future, even if other clients desire it.

MRocklin
  • 55,641
  • 23
  • 163
  • 235
  • Thanks. How easy can it be... :-) And how about the second part of my question? If I have submitted a long dsk using fire_and_forget, and I cancel the final task, will that also cancel the rest of the graph? Or do I need to cancel every single task separately? – Vincent Schut Sep 20 '17 at 07:40
  • Is is possible to design a long-running task that I can cancel once it has started executing? I understand that it is difficult to kill a python thread, but can I instead message a task to shutdown cleanly using either metadata or signals? – sheridp Mar 15 '18 at 19:28
  • See https://stackoverflow.com/questions/49203128/how-do-i-stop-a-running-task-in-dask – MRocklin Mar 15 '18 at 19:39