5

I trying to deploy a django project using django, but I have these questions unsolved:

  • Should I run one celeryd for each web server?
  • Should I run just one RabbitMQ server, on another machine (not) running celeryd there, accesible to all my web servers? or RabbitMQ must be run also on each of the web servers?
  • How can I use periodic tasks if the code is the same in all web servers?

Thank for your answers.

Arkaitz
  • 51
  • 2

1 Answers1

6

It really depends on the size of the project, ideally you have RabbitMq, celery workers and web workers running on different machines.

You need only one RabbitMQ and eventually multiple queue workers (bigger queues need more workers of course).

You dont need 1 celery worker per webworker, the webworkers are going to publish tasks to the broker and then the workers will get them from there, in fact the webworker does not care about the amount of workers connected to the broker as it only communicates with the broker.

Of course if you are starting a project it makes sense to keep everything on the same hardware and keep budget low and wait for the traffic and the money to flow :)

You want to have the same code on every running instance of your app, no matter if they are celery workers/ webservers or whatever.

Tommaso Barbugli
  • 11,781
  • 2
  • 42
  • 41
  • What should I do to have each worker read the same queue? I spent many hours search for an example, but nothing found. Thanks. – user2372074 Oct 10 '14 at 07:22