1

I'm using a queue channel with queue capacity of 1000, what I'd like to know. When my application is being restarted, what will happen to those messages which are being stored in my queue. Will there be a message loss? Is it possible to handle this scenario without message loss.

user14566279
  • 49
  • 3

1 Answers1

0

To not lose messages in between, you need to configure a persistent message store for your queue channels.

See more info in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/system-management.html#message-store

Also read this article where we give a big accent to the persistent store to keep in mind: https://spring.io/blog/2020/10/26/case-study-aggregator-function-and-processor

Artem Bilan
  • 113,505
  • 11
  • 91
  • 118
  • Is it possible to go with File based message storage rather than db based message store? – user14566279 Nov 09 '20 at 17:27
  • Well, technically any database ends up in files, so I'm not sure what is your point. You also may consider to have some messaging middleware-based channel - JMS, Kafka, RabbitMQ, Redis etc. – Artem Bilan Nov 09 '20 at 17:40
  • I'm stopping and starting the application multiple times and randomly message count in message store is getting reduced. Do we need to set any parameters? – user14566279 Nov 10 '20 at 14:14
  • Not sure what you are looking for. It is indeed normal that your started application is consuming messages from the queue channel, so the "message count in message store is getting reduced". – Artem Bilan Nov 10 '20 at 15:20
  • I'm using redis message store for saving message in spring integration. While restarting the application, occassionally I could see the number of messages in the store is being reduced. There is no transactions being done but still the message count is being reduced after multiple restart. – user14566279 Nov 10 '20 at 15:31
  • Not sure why is the question. What you describe is natural for integration solution when you have such a `QueueChannel` and `PollingConsumer` on it. So, when you start the application and there are some messages in the queue, they are consumed by the poller according its interval: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#endpoint-pollingconsumer – Artem Bilan Nov 10 '20 at 15:35
  • Exactly they are being consumed. But I'm handling a negative scenario where message will not get delivered, an exception will be thrown and message gets retained back in the message store. Occasionally after multiple restart there is a loss of one message – user14566279 Nov 10 '20 at 15:38
  • Who said you that it is going to work that way? It is possible only if a persistent store is transactional. At the moment it is possible only with `JdbcChannelMessageStore` – Artem Bilan Nov 10 '20 at 15:40
  • Or as I said before: use messaging middleware, like ActiveMQ, RabbitMQ. Kafka is good, too. With all others you are on your own when exception happens. You may consider to implement an error handling for those exceptions and manual redelivery back to the same queue channel. – Artem Bilan Nov 10 '20 at 15:42
  • Even in the exception scenario, I'm sending the message back to the same channel. – user14566279 Nov 10 '20 at 15:52
  • I don't see this issue often. Occasionally after multiple times of re-start it gets lost. Fixed delay is 1000milli second and maxmsg per poll is 100. Do I hav to enable any specific property? – user14566279 Nov 10 '20 at 15:54
  • What has that property do for you? What are you looking for? – Artem Bilan Nov 10 '20 at 16:02
  • In exception flow messages are being sent to the same channel.lets say I've 5 message in message store. After multiple restart the count gets reduced by 1. I'm not sure about the reason for reduction in count. Appreciate if you could provide your insights – user14566279 Nov 10 '20 at 16:04
  • There is no such a property to set. You have to implement your own error handling and send failed messages back to the same channel. Consider to not use that "maxmsg per poll is 100". Just `1` to be as safe as possible. – Artem Bilan Nov 10 '20 at 16:05