0

I have a Reactive Spring Boot application consuming messages from RabbitMQ and persisting them in a (MongoDB) repository:

@RabbitListener(...)
public void processMessage(Message message) {
    repository.persist(message).subscribe();
}

Assuming multiple messages arriving in a short period of time, this code could deplete a configured ConnectionPool to the database. If I would receive the messages in a Flux, I could concatMap() them into the db or insert them in buckets of n documents.

That's why I tried to implement a bridge of the given RabbitMQ listener to a self-managed Flux:

@Component
public class QueueListenerController {

    private final MyMongoRepository repository;
    private final FluxProcessor<Message, Message> fluxProcessor;
    private final FluxSink<Message> fluxSink;

    public QueueListenerController(MyMongoRepository repository) {
        this.repository = repository;
        this.fluxProcessor = DirectProcessor.<Message>create().serialize();
        this.fluxSink = fluxProcessor.sink();
    }

    @PostConstruct
    private void postConstruct() {
        fluxProcessor.concatMap(repository::persist)
                .subscribe();
    }

    @RabbitListener(bindings = @QueueBinding(
            value = @Queue(value = "my-queue", durable = "true", autoDelete = "false"),
            exchange = @Exchange(value = "amq.direct", durable = "true", autoDelete = "false")
    ))
    public void processMessage(Message message) {
        fluxSink.next(message);
    }
}

This works locally and for a certain period of time but after some time (I expect 12-24 hours) it stops storing messages in the database, so I'm quite sure I'm doing something wrong.

What would be the correct way of transforming incoming RabbitMQ messages into a Flux of messages?

user3105453
  • 1,881
  • 5
  • 32
  • 55
  • Are you not using the reactive repositories? – 123 May 26 '20 at 21:25
  • How are you acknowledging the messages? If auto; you risk message loss when async processing messages passed to the `@RabbitListener`. If you are using MANUAL acks; perhaps you are missing some - when you hit the prefetch of unack'd messages you won't get any more. However, I don't see you passing the channel to the listener, so perhaps you are acking none? – Gary Russell May 26 '20 at 22:23
  • Thanks, @GaryRussell for your comment. I've added the parameters of the `@RabbitListener` to the question. 123, I do use reactive repositories, hence the 'subscribe()' :-) – user3105453 May 27 '20 at 06:57
  • Consider using https://github.com/reactor/reactor-rabbitmq instead of spring-rabbit. – Gary Russell May 27 '20 at 13:37
  • I considered it but I couldn't find out how to configure it to use the injected rabbitmq-connection details of Cloud Foundry. That's why I'd like to rely on Spring Rabbit and not touch it. But of course, if your answer includes a snippet how to use it properly, I'm more than happy to accept that. – user3105453 May 27 '20 at 19:54

0 Answers0