I have a task that I would like to run every hour. I need to run it 3 times (for every element in list) but instead of having them run one after another I would like them to run all at once. How is this possible with Celery? I am using Flask as well.
Right now, I have a file called tasks.py with my tasks, and I use a celerybeat schedule to run every hour.
This is the function I call every hour:
Note that I am calling a function I import into the file.
@celery.task
def stats():
list = [1,2,3]
functions = [sensor_stats(60, each) for each in list]
results = [function.wait() for function in functions]
Update: I defined a new task, that would loop through the list and create a group that would call the stats task for each list element, that way each one is running in parallel.
See http://celery.readthedocs.org/en/latest/reference/celery.html#celery.group for more info
It looked something like this:
@celery.task
def stats(interval, f_id):
sensor_stats(interval, f_id)
@celery.task
def sensor_statistics():
list = [1,2,3]
res = group
group(stats_function.s(60, each) for each in list)()
CELERYBEAT_SCHEDULE = {
'Stat Script': {
'task': 'tasks2.sensor_statistics',
'schedule': timedelta(hours=1),
'args': (),
}
}
