RabbitMQ memory management with Docker

Question

I'm using the RabbitMQ management docker image. The issue is after around 1 week uptime, the disk space and the memory gets almost drained. I need to again terminate and restart it to function normally, but by doing so, all the existing messages in the queues are lost.

Command I use to start the rabbitmq server is : docker run --rm -it --hostname my-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management

I'm using persistent messages and durable queues.

No. of queues will be around 70.

Connections per second may vary from 5 to 100

I'm using amqplib npm module to interact with RabbitMQ using Nodejs.

Am I doing something wrong? Or should I configure on any server directly instead if using docker image? Is there any way to delete only the acknowledged messages?

Thanks in advance.

Answer 1

this is the definition of durability according to rabbitmq official documentation:

Durability

Queues can be durable or transient. Metadata of a durable queue is stored on disk, while metadata of a transient queue is stored in memory when possible. The same distinction is made for messages at publishing time in some protocols, e.g. AMQP 0-9-1 and MQTT.

In environments and use cases where durability is important, applications must use durable queues and make sure that publish mark published messages as persisted.

so if you are using durability in the messages section it means you are storing them to storage. rabbitmq comes with a functionality that you can ack your message. if you ack the message it will be removed from the queue. so you have two options here for ack:

1. use auto ack mode { noAck: true } :

   channel.consume(queue, function(msg) { // the message it self is here! }, { noAck: true });

2. ack manualiy:

   channel.consume(queue, function(msg) { // the message it self is here! // I can ack manually channel.ack(msg); }, { noAck: false });

so there is an article for optimization of rabbitmq: https://www.cloudamqp.com/blog/2018-01-08-part2-rabbitmq-best-practice-for-high-performance.html