There is a tracker class, that just counts success, failed, pending, and started tasks via redis.
The goal is to extend Celery, so its workers can access the group_id and keep statistics for the group. I expect an interface similar to:
def on_important_event(...):
group_id=uuid4()
for _ in range(count_of_jobs):
my_task.apply_async(..., group_id=group_id)
custom Task class would look like:
class MyTask(Task):
# declaring group_id somehow
def apply_async(...):
get_tracker(self.request.group_id).task_pending()
...
def before_start(...):
get_tracker(self.request.group_id).task_started()
...
def on_success(...):
get_tracker(self.request.group_id).task_success()
...
def on_failure(...):
get_tracker(self.request.group_id).task_failed()
...
I could not find a way to implement the class so it will properly save and receive custom attribute through AMQP.
UPD, to make clear:
The problem is to mark some calls of Tasks as a participant of a group. So I can track the group not general Task or a single call.
As it seems to me, there must be a way to add an attribute for Task that would be saved into Queue and then received by a Celery's worker so i can access it on Task class layer.
