2

I'm using a SimpleMessageListenerContainer as a basis for remoting over AMQP. Everything goes smooth provided that the RabbitMQ broker can be reached at process startup. However, if by any reason it can't be reached (network down, permissions problem, etc...) the container just keeps retrying to connect forever. How can I set up a retry behaviour in this case (for example, try at most 5 times with an exponential backoff and then abort, killing the process)? I've had a look at this, but it doesn't seem to work for me on container startup. Can anyone please shed some light?

At the very least, I'd like to be able to catch the exception and provide a log message, instead of printing the exception itself as is the default behaviour.

Community
  • 1
  • 1
Martin
  • 1,317
  • 3
  • 13
  • 18
  • Would you mind sharing the StackTrace on the matter? There is no such an option to `retry` consumer start. That `Advice[]` is for the target listener - business operation, not that technical startup. – Artem Bilan Feb 05 '15 at 12:01

1 Answers1

4

How can I set up a retry behaviour in this case

There is no sophisticated connection retry, just a simple recoveryInterval. The assumption is that the broker unavailability is temporary. Fatal errors (such as bad credentials) stop the container.

You could use some external process to try connectionFactory.createConnection() and stop() the SimpleMessageListenerContainer when you deem it's time to give up.

You could also subclass CachingConnectionFactory, override createBareConnection catch the exception and increment the recoveryInterval, then call stop() when you want.

EDIT

Since 1.5, you can now configure a backOff. Here's an example using Spring Boot...

@SpringBootApplication
public class RabbitBackOffApplication {

    public static void main(String[] args) {
        SpringApplication.run(RabbitBackOffApplication.class, args);
    }

    @Bean(name = "rabbitListenerContainerFactory")
    public SimpleRabbitListenerContainerFactory simpleRabbitListenerContainerFactory(
            SimpleRabbitListenerContainerFactoryConfigurer configurer,
            ConnectionFactory connectionFactory) {
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        configurer.configure(factory, connectionFactory);
        BackOff recoveryBackOff = new FixedBackOff(5000, 3);
        factory.setRecoveryBackOff(recoveryBackOff);
        return factory;
    }

    @RabbitListener(queues = "foo")
    public void listen(String in) {

    }

}

and

2018-04-16 12:08:35.730  INFO 84850 --- [           main] com.example.RabbitBackOffApplication     : Started RabbitBackOffApplication in 0.844 seconds (JVM running for 1.297)
2018-04-16 12:08:40.788  WARN 84850 --- [cTaskExecutor-1] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it. Exception summary: org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
2018-04-16 12:08:40.788  INFO 84850 --- [cTaskExecutor-1] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer@57abad67: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
2018-04-16 12:08:40.789  INFO 84850 --- [cTaskExecutor-2] o.s.a.r.c.CachingConnectionFactory       : Attempting to connect to: [localhost:1234]
2018-04-16 12:08:45.851  WARN 84850 --- [cTaskExecutor-2] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it. Exception summary: org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
2018-04-16 12:08:45.852  INFO 84850 --- [cTaskExecutor-2] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer@3479ea: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
2018-04-16 12:08:45.852  INFO 84850 --- [cTaskExecutor-3] o.s.a.r.c.CachingConnectionFactory       : Attempting to connect to: [localhost:1234]
2018-04-16 12:08:50.935  WARN 84850 --- [cTaskExecutor-3] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it. Exception summary: org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
2018-04-16 12:08:50.935  INFO 84850 --- [cTaskExecutor-3] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer@2be60f67: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
2018-04-16 12:08:50.936  INFO 84850 --- [cTaskExecutor-4] o.s.a.r.c.CachingConnectionFactory       : Attempting to connect to: [localhost:1234]
2018-04-16 12:08:50.938  WARN 84850 --- [cTaskExecutor-4] o.s.a.r.l.SimpleMessageListenerContainer : stopping container - restart recovery attempts exhausted
Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Is it still possible to subclass `CachingConnectionFactory` and override `createBareConnection`? Because as of version 1.7.4 of Spring AMQP methods for creating a connection are final. How could we do it now? – djeison Apr 16 '18 at 13:53
  • Oops - my bad - it's always been `final`. However, since 1.5, you can now configure a `recoveryBackOff` which can be used to achieve the desired behavior (including stopping the container when the `nextBackOff()` returns `STOP`). – Gary Russell Apr 16 '18 at 14:03
  • Thank you for your reply. Would you know of any tutorial on how to configure this `recoveryBackOff`? I have tried declaring a `SimpleRabbitListenerContainerFactory` bean and setting a `FixedBackOff` to it but it seems to ignore my config of `maxAttempts` = 3 and keeps on trying to connect after many connection failures. – djeison Apr 16 '18 at 14:54
  • If you can ask a new question, I'll answer it with a sample. – Gary Russell Apr 16 '18 at 16:03
  • 1
    I edited my answer with an example. `stopping container - restart recovery attempts exhausted`. – Gary Russell Apr 16 '18 at 16:12
  • Ok, thank you very much for your example, I followed it here and it has indeed stopped retrying connecting after reaching the number of `maxAttempts`. The only thing now is that it keeps trying to run "Health checks" against RabbitMQ therefore causing "eternal" `org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused` in my console. How could we handle it so it fails silently? – djeison Apr 16 '18 at 16:32
  • I have asked a proper question here: https://stackoverflow.com/questions/49862520/rabbitmq-spring-boot-application-handle-broker-down I appreciate if you may keep on helping – djeison Apr 16 '18 at 16:55