A brief overview
I have a microservice architecture powered by Python 3.6+ where it processes an upwards of 10 million transactions per day through RabbitMQ.
A significant part of the architecture is a set of 16-32 parallel consumers launched as multiprocessing.Process instances that fetch and process these transactions from RabbitMQ.
The library currently used for interfacing with RabbitMQ through Python is pika
What are the issues and challenges?
The BlockingConnection adapter of pika is rather unreliable. I believe it maintains its own polling loop. Under a sudden burst of transactions, the connections get dropped.
I have retries configured in a jitter, exponential back-off model through the tenacity Python library, watching on the base exception pika.exceptions.AMQPError.
But in spite of this safeguard, the connections are dropped abruptly:
104:Connection Reset by Peer
or
in basic_publish
raise exceptions.ChannelClosed()
pika.exceptions.ChannelClosed
Looking at the Asynchronous implementations of pika consumer and publisher,
- it looks like a tangled up mess of callbacks with no properly documented sequence
- the added complexity that I assume of running this event loop based consumer in a separate thread in a multiprocessing based architecture and communicating back the results to the process that actually needs it
The question being: What is a stabler alternative to pika?
I came across amqp. Their homepage mostly has an API reference without any usage patterns or examples documented.
Is this a stable enough library to dig into? Or are there any better alternatives?
