kombu not reconnecting to RabbitMQ

Question

I have two servers, call them A and B. B runs RabbitMQ, while A connects to RabbitMQ via Kombu. If I restart RabbitMQ on B, the kombu connection breaks, and the messages are no longer delivered. I then have to reset the process on A to re-establish the connection. Is there a better approach, i.e. is there a way for Kombu to re-connect automatically, even if the RabbitMQ process is restarted?

My basic code implementation is below, thanks in advance! :)

def start_consumer(routing_key, incoming_exchange_name, outgoing_exchange_name):
    global rabbitmq_producer

    incoming_exchange = kombu.Exchange(name=incoming_exchange_name, type='direct')
    incoming_queue = kombu.Queue(name=routing_key+'_'+incoming_exchange_name, exchange=incoming_exchange, routing_key=routing_key)#, auto_delete=True)

    outgoing_exchange = kombu.Exchange(name=outgoing_exchange_name, type='direct')
    rabbitmq_producer = kombu.Producer(settings.rabbitmq_connection0, exchange=outgoing_exchange, serializer='json', compression=None, auto_declare=True)

    settings.rabbitmq_connection0.connect()
    if settings.rabbitmq_connection0.connected:
        callbacks=[]
        queues=[]

        callbacks.append(callback)
        # if push_queue:
        #   callbacks.append(push_message_callback)
        queues.append(incoming_queue)

        print 'opening a new *incoming* rabbitmq connection to the %s exchange for the %s queue' % (incoming_exchange.name, incoming_queue.name)
        incoming_exchange(settings.rabbitmq_connection0).declare()
        incoming_queue(settings.rabbitmq_connection0).declare()

        print 'opening a new *outgoing* rabbitmq connection to the %s exchange' % outgoing_exchange.name
        outgoing_exchange(settings.rabbitmq_connection0).declare()

        with settings.rabbitmq_connection0.Consumer(queues=queues, callbacks=callbacks) as consumer:
            while True:
                settings.rabbitmq_connection0.drain_events()

It's not for kombu, but I recently wrote a robust consumer example for my own library. If anything you could probably adopt it for kombu/pika. https://github.com/eandersson/amqpstorm/blob/stable/examples/scalable_consumer.py — eandersson, Jun 01 '16 at 21:27

score 0 · Answer 1 · answered Aug 09 '16 at 14:17

On the consumer side, kombu.mixins.ConsumerMixin handles reconnecting when the connection goes away (and also does heartbeats, etc., and lets you write less code). There doesn't seem to be a ProducerMixin, unfortunately but you could potentially dig into the code and adapt it...?

kombu not reconnecting to RabbitMQ

1 Answers1