Too many consumers in Rabbit MQ (Bunny)

Question

I'm sending lots of data to my app through JMeter.

My subscribe block and the publisher look like this:

    BunnyStarter.start_bunny_components
    cons = BunnyStarter.queue.subscribe do |delivery_info, metadata, payload|
      method_calling ( payload )
      cons.cancel
    end

    BunnyStarter.exchange.publish(body.to_json, routing_key:  BunnyStarter.queue.name)

And my BunnyStarter class:

def self.start_bunny_components
  if @@conn.nil?

    @@conn = Bunny.new
    @@conn.start

    @@ch = @@conn.create_channel

    @@queue  = @@ch.queue("dump_probe_queue")
    @@exchange = @@ch.default_exchange
  end
end

The problem is, although I call consumer.cancel after method_calling, in my Rabbit MQ admin I still see that I get like one thousand consumers created in about 6 minutes.

Is that because of the rate and the amount of data I'm sending?

How can I improve this?

score 0 · Answer 1 · answered Sep 06 '16 at 16:57

I have seen this issue before. The reason its creating 1000 of consumers is because you are creating channel per connection. Eventually your consumer will shut down after a while because of this.

The number of consumers getting created are not because of the data but its because in the consumer its creating one connection per subscription.

Solution: Instead of creating multiple channels, create only one channel and use number of connections using the same channel. I mean one instance of Connection and multiple instances of Model so you can share the same connection for multiple model.

Too many consumers in Rabbit MQ (Bunny)

1 Answers1