Flask Celery Redis

Question

I have a python class. When it is initialized, a browser window is created using selenium. There is also an array of objects, the elements of which I want to throw into the queue. Is it possible to assign an entity to workers - objects of this class, in order to subsequently call its method, where will the elements of the queue be passed?

@celery_app.task
def process_road(road_data):
    window = Window(road_data)  //my class
    window.collect_photo()
    return True


def process(road_data):
    batch_size = len(road_data) // 5
    batches = [road_data[i:i+batch_size] for i in range(0, len(road_data), batch_size)]
    task_ids = []
    for batch in batches:
        task = process_road.delay(batch)
        task_ids.append(task.id)
    return jsonify({'task_ids': task_ids}), 202

Here I immediately create 5 workers and give each one my portion of the array elements. I know it's wrong

score 0 · Answer 1 · answered Jul 26 '23 at 17:32

You can only pass serialisable objects by pickle or json in your task arguments. Celery internally serialises the task args and kwargs and sends them over the broker for the worker to execute it. So, it's not possible to directly send your selenium class as the celery argument.

Flask Celery Redis

1 Answers1