2

I was wondering if there's a way running tasks asynchronously that will run in the background (Using Celery for example) which will never run simultaneously?

Which means, each task can run by itself simultaneously with it self but not with another tasks that interfere with the actions of the first task.
For example,

  • Task A: Reads from a file (can run simultaneously with it self (with other tasks that reads from files)
  • Task B: Writes to a file (Should not run simultaneously with the read tasks (With task A))

Essentially, what I need is a way for tasks A and B to find out if the other task is running and if it is, then delay itself and wait until it's done (probably with blocking the task queue)

Does defining a queue for the tasks solves the problem? or is it just a queue for the execution of tasks (So it will execute the 2nd task in the queue without waiting for the result of the first one)?

Is using a lock my only solution here?

If the lock solution is the only one, what's the correct way of implementing this?
I have found this: Ensuring a task is only executed one at a time
But it uses django's cache as a lock and I'm not running my programs in a django environment so it doesn't work for me.

Drxxd
  • 1,860
  • 14
  • 34
  • Define `task`. Because the most obvious answer would be to have a single consumer thread fetch "tasks" as items from a queue, and prcess them one at a time. – spectras Oct 26 '16 at 15:59
  • Task = set of defined actions like reading from file or writing into a DB... Does Celery or any other task queuing systems provide a consumer that process tasks one at a time? Or I'll have to implement it myself? – Drxxd Oct 26 '16 at 16:10
  • Could you specify the scope of your problem bit more? What are you running? On how many machines? When should it run? – grepe Oct 26 '16 at 16:23
  • One machine, I'm implementing a server app which performs specific tasks on specific requests, the requests are handled asynchronously and the tasks are handled in a delay (using celery) (after responding to the clients). Now there are two tasks which must not run at the same time.. Let's say that one of them registers connections to specific files and the other one notifies the registered connections about state changes on these files.. The registered client must get a response as soon as the state changes (without locks the state could change and the client won't receive a notification) – Drxxd Oct 26 '16 at 16:39

0 Answers0