Compressing and decompressing Spring RabbitMQ messages with a DirectMessageListenerContainer

Question

I have modified my RabbitMQ from a previous post (spring-rabbit JSON deserialization for ArrayList contents) to now use a DirectMessageListener with MessagePostProcessors to GZip and GUnzip the message payloads.

However, it doesn't appear to be working as the breakpoints are not activated, but also because my RabbitListeners are no longer receiving messages, whereas they did with a SimpleMessageFactoryListenerContainer. Also, it appears the SimpleMessageListenerContainer(?) is still being used. On a side-note, I am autowiring the DirectMessageListenerContainer so I can dynamically set the queues I used.

---

spring-rabbit: 2.0.3.RELEASE. spring-boot: 2.0.1.RELEASE.

---

RabbitMQ configuration:

@Configuration
@EnableRabbit
public class MessagingConfiguration implements ShutdownListener {

    @Autowired
    private RabbitListenerEndpointRegistry registry;

    @Autowired
    private DirectMessageListenerContainer container;

    @Bean
    public DirectMessageListenerContainer messageListenerContainer(final ConnectionFactory connectionFactory) {
        final DirectMessageListenerContainer listenerContainer = new DirectMessageListenerContainer();
        listenerContainer.setConnectionFactory(connectionFactory);
        listenerContainer.setMessageConverter(jsonConverter()); // i.e.@RabbitListener to use Jackson2JsonMessageConverter
        listenerContainer.setAutoStartup(false);
        // container.setErrorHandler(errorHandler);
        final MessageListenerAdapter messageListener = new MessageListenerAdapter(new Object() {
            @SuppressWarnings("unused")
            public String handleMessage(final String message) {
                return message.toUpperCase();
            }
        });
        messageListener.setBeforeSendReplyPostProcessors(new GZipPostProcessor());
        listenerContainer.setMessageListener(messageListener);
        listenerContainer.setAfterReceivePostProcessors(new GUnzipPostProcessor());
        return listenerContainer;
    }

    @EventListener(ApplicationDatabaseReadyEvent.class)
    public void onApplicationDatabaseReadyEvent() {
        log.info("Starting all RabbitMQ Listeners..."); //$NON-NLS-1$
        for (final MessageListenerContainer listenerContainer : registry.getListenerContainers()) {
            listenerContainer.start();
        }
        log.info("Register is running: {}", registry.isRunning()); //$NON-NLS-1$
        log.info("Started all RabbitMQ Listeners."); //$NON-NLS-1$
    }

    @Bean
    public List<Declarable> bindings() {
    final List<Declarable> declarations = new ArrayList<>();
        final FanoutExchange exchange = new FanoutExchange("fx", true, false);
        final Queue queue = QueueBuilder.durable("orders").build();
        declarations.add(exchange);
        declarations.add(queue);
        declarations.add(BindingBuilder.bind(queue).to(exchange));
        List<String> q = new ArrayList<>();
        q.add(queue.getName());
    container.addQueueNames(q.toArray(new String[queues.size()]));

        return declarations;
    }

    @Bean
    public Jackson2JsonMessageConverter jsonConverter() {
        final Jackson2JsonMessageConverter converter = new Jackson2JsonMessageConverter();
        converter.setClassMapper(classMapper());
        return converter;
    }

    private static DefaultJackson2JavaTypeMapper classMapper() {
        final DefaultJackson2JavaTypeMapper classMapper = new DefaultJackson2JavaTypeMapper();
        classMapper.setTrustedPackages("*"); //$NON-NLS-1$  //TODO add trusted packages
        return classMapper;
    }

    @ConditionalOnProperty(name = "consumer", havingValue = "true")
    @Bean
    public ConsumerListener listenerConsumer() {
        return new ConsumerListener();
    }

    @ConditionalOnProperty(name = "producer", havingValue = "true")
    @Bean
    public ProducerListener listenerProducer() {
        return new ProducerListener();
    }

    @Bean
    public RabbitAdmin rabbitAdmin(final CachingConnectionFactory connectionFactory) {
        return new RabbitAdmin(connectionFactory);
    }

    @Bean
    public RabbitTemplate rabbitTemplate(final ConnectionFactory connectionFactory) {
        final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
        rabbitTemplate.setMessageConverter(jsonConverter()); // convert all sent messages to JSON
        rabbitTemplate.setReplyTimeout(TimeUnit.SECONDS.toMillis(3));
        rabbitTemplate.setReceiveTimeout(TimeUnit.SECONDS.toMillis(3));
        return rabbitTemplate;
    }

    @Override
    public void shutdownCompleted(final ShutdownSignalException arg0) {
    }
}

Answer 1

It doesn't work that way, you can't autowire containers for @RabbitListeners; they are not beans; they are created by the container factory and registered in the registry. Instead you have to retrieve them from the registry (by id).

However, since you have autoStartup set to false, it shouldn't be "stealing" messages from your @RabbitListener.

Generally, DEBUG logging should help.

Answer 2

The official guide explains it at https://docs.spring.io/spring-amqp/reference/html/#post-processing

Basically there are couple of options i.e.

1. Configure Message convertor like below

   public class CompressionMessageConverter {
   public static MessageConverter messageConverter() {
       final MessageConverter simpleMessageConverter = new SimpleMessageConverter();
       return new MessageConverter() {
           @Override
           public Message toMessage(Object messagePayload, MessageProperties messageProperties)
                   throws MessageConversionException {
               final Message m = simpleMessageConverter.toMessage(messagePayload, messageProperties);
               final ByteArrayOutputStream os = new ByteArrayOutputStream();
               try (GZIPOutputStream gz = new GZIPOutputStream(os)) {
                   gz.write(m.getBody());
               } catch (IOException e) {
                   throw new MessageConversionException(e.getMessage());
               }
   
               return new Message(os.toByteArray(), messageProperties);
           }
   
           @Override
           public Object fromMessage(Message messagePayload) throws MessageConversionException {
               final ByteArrayOutputStream os = new ByteArrayOutputStream();
               try (GZIPInputStream is = new GZIPInputStream(new ByteArrayInputStream(messagePayload.getBody()))) {
                   byte[] b = new byte[2048];
                   int r;
                   while ((r = is.read(b)) != -1) {
                       os.write(b, 0, r);
                   }
               } catch (IOException ioException) {
                   throw new RuntimeException(ioException);
               }
   
               return simpleMessageConverter.fromMessage(new Message(os.toByteArray(), messagePayload.getMessageProperties()));
           }
       };
   }
   

Use it in RabbitTemplate as

rabbitTemplate.setMessageConverter(CompressionMessageConverter .messageConverter());

2. Use post processors

   rabbitTemplate.setBeforePublishPostProcessors(new GZipPostProcessor()); rabbitTemplate.setAfterReceivePostProcessors(new GUnzipPostProcessor());