Making spring amqp consumer to stop consuming messages for a specified type of exception

Question

I am using spring amqp using rabbitmq.

I have a use case where my consumer relies of another system - X which may have downtimes. What I would want is to handle the exception I get when X is down in the following way - On XDownException, I would like to stop processing the messages from the queue so that I don't lose these messages during the downtime and keep requeueing the message until I stop getting XDownException. This way, I am sure that I would not lose any messages when X is down and then resume automatically as soon as X is up.

Fifo is a must. The listener is thrown XDownException when processing the message. The listener is not transaction aware right now, but we can make it transaction aware if that helps. However I don't want to do this for every kind of exception... Is there a way I can do this with spring amqp?

Also, is there a better way to accomplish this than this approach? I don't have an event for when X comes up.

Question 1: Presuming you are using message-driven POJO with a spring listener container, do you expect XDownException during processing of received message in your listener (along with possible other exception)? Question 2: Is your listener transaction aware? Question 3: is FIFO (in other words, guaranteed sequential delivery) a must requirement for you? Solution to your use case will depend on answers to these question. Please update you question with additional inputs in order for others to help. — Avnish, Jul 04 '20 at 16:50

score 1 · Answer 1 · answered Jul 05 '20 at 07:13

FIFO requirement dictates that you must have no more than one concurrent consumer in your Container setup. Presuming this setup, you'll receive each message one by one in your POJO method. Next message wouldn't be delivered unless this message is fully processed.

Following is what the strategy should be to in this case for the use case you described.

Make sure you are set prefetch size to 1 and manual acknowledge mode on your container. Full details available at Asynchronous Consumer
Your setup should look like following


public class ExtendedListenerAdapter extends MessageListenerAdapter {
    @Override
    protected Object[] buildListenerArguments(Object extractedMessage, Channel channel, Message message) {
        return new Object[]{extractedMessage, channel, message};
    }
}


public class MyListener {
    public void handleMessage(Object object, Channel channel, Message message) throws IOException {
        try {
            processMessage(Object)
        } catch (XDownException xdex) {
            waitForXAvailability(object);
        } catch (OtherException oex) {
        } finally {
            channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
        }
    }
    
    private processMessage(Object object) /* throws All types of exceptions */ {
        // process the message as usual
    }
    
    private waitForXAvailability(Object object) {
        for (;;) {
            // add delay here, exponetial backoff delay recommend with upper bound
            // also add upper bounds on number of iteration you want to keep, it's infinte now
            try {
                processMessage(Object);
                return; // no exception means X is up again
            } catch (XDownException xdex) {
                // x is down, let the loop continue
            }
        }
    }
}

@Configuration
public class ExampleAmqpConfiguration {

    @Bean
    public SimpleMessageListenerContainer messageListenerContainer() {
        SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
        container.setConnectionFactory(rabbitConnectionFactory());
        container.setQueueName("some.queue");
        container.setPrefetchCount(1)
        container.setAcknowledgeMode(AcknowledgeMode.MANUAL)
        container.setMessageListener(myAdapter());
        return container;
    }

    @Bean
    public ConnectionFactory rabbitConnectionFactory() {
        CachingConnectionFactory connectionFactory =
            new CachingConnectionFactory("localhost");
        connectionFactory.setUsername("guest");
        connectionFactory.setPassword("guest");
        return connectionFactory;
    }

    @Bean
    public ExtendedListenerAdapter myAdapter() {
        ExtendedListenerAdapter adapter = new ExtendedListenerAdapter();
        listenerAdapter.setDelegate(myListener())
        return adapter;
    }
    
    @Bean
    public MyListener myListener() {
        return new MyListener();
    }
}

Please tune above outline to best suit your needs. Following links should give you additional helpful information

Thanks for the answer... I see that we are retrying the process message until X is back up again. Adding a delay helps optimize the number of retries. I have up voted the answer, but I am just thinking if there is a better way to solve this. — Ankit Gupta, Jul 05 '20 at 09:17
There are better ways but strict FIFO requirement makes the situation complicated here. This is further complicated by the fact that there's no explicit notification of X availability thus retry is the only option to identify if X is available or not. Had there been an event which notifies X is up again, then [pausing and resuming listener](https://stackoverflow.com/questions/55836140/how-to-pause-resume-individual-spring-jms-message-listeners) would have been an option as well. — Avnish, Jul 05 '20 at 09:25
Why need manual acknowledgment if listener returns normally with ack? — Anatoly Utkin, Feb 28 '23 at 20:27

Making spring amqp consumer to stop consuming messages for a specified type of exception

1 Answers1