1

I'm trying to understand what is happening between Web-Dynos, Worker-Dynos and RabbitMQ on my Heroku-hosted backend.

I notice that I have over 10 connections to RabbitMQ, although I so far I'm using one Web-Dyno and one Woker-Dyno. I am using greenthreads (eventlet) and expect to have one connection per Web- or Worker-Dyno.

This is my worker's start from Procfile: celery -E -A proj worker -B -l info --maxtasksperchild=1000

In my case I can have over 10 open connections: enter image description here

Here is also list of my queues (celery seems to be used for queuing the tasks, not quite sure what the other two are for): enter image description here

I've tried playing with BROKER_POOL_LIMIT and it does not seem to help. Right now BROKER_POOL_LIMIT = 1 and the number of connections is 10.

Can it be caused by having eventlet instead of usual gunicorn-workers?

silentser
  • 2,083
  • 2
  • 23
  • 29
  • 1
    You have to limit the number of workers with `-w` and the concurrency with `-c` – Carl Hörberg Oct 29 '14 at 09:05
  • I have concurrency set to 4 processes (defaults to the number of CPUs) and have 4 web workers. If I use BROKER_POOL_LIMIT * (gunicorn-workers * web dynos + worker dynos * concurrency) formula, I should never get more than 1*(1*4 + 1*4)=8 connections. – silentser Oct 30 '14 at 09:55
  • 1
    Not entirely sure, but this could be due to mingle and gossip. Did you try --without-mingle and --without-gossip? Also, curious to know, which monitoring tool is this? – arjunattam Apr 05 '16 at 13:29
  • @arjun27 This monitoring tool offered by AMPQ - a RabbitMQ SAAS provider – silentser Apr 06 '16 at 08:02

0 Answers0