-1

I need to build a rest API for scheduling tasks I was using celery with flask-restful up until not but celery does not support pausing tasks in the middle from the clients. Is this possible with Celery in a way that I can't figure out or do any other task queue support this?

pranaay Saini
  • 1
  • 1
    Can you give some examples of tasks ? What happens if these tasks are paused ? Do you need to be able to stop the tasks immediately and at any time ? – Daniser Jul 31 '20 at 19:07
  • its long-running tasks like uploading/processing a CSV file. I want to be able to pause these in the middle and then resume whenever I want. Yes I want to be able to stop tasks at any time . – pranaay Saini Aug 01 '20 at 22:14

1 Answers1

0

In order to prevent data corruption (like missing/invalid row in CSV file), you can split each task to multiple small "sub-tasks" (e.g: processing a single row of a CSV file can be a sub-task).

If a user stops you finish the current sub-task but don't continue to the next sub-task (and save the id/index of the last sub-task that finished successfully).

When the user wants to continue you can pick up where you left off (start processing more sub-tasks until you are done or until the user stops again).

If you make the sub-tasks small/short/quick enough, the user will "feel" as if the task stopped immediately.

Daniser
  • 162
  • 4