0

I want to create a simple IntegrationFlow with Spring integration, and I am having difficulties.

I want to create an integration flow that takes messages from multiple queues in Rabbit Mq and posts the messages to different Rest endpoints.

i want to know if i can parallelize this.

i have two scenarios that i want to check the feasibility :

  • the first one i want to create a thread for every RabbitMq Queue that would listen and execute the flow after receiving a message :

Scenario 1

  • the second scenario : in this scenario i want to create a dynamic number of threads for every queue , the number of threads goes up or down depending on the number of messages.

Scenario 2 

 HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        RestTemplate restTemplate = new RestTemplate();
        SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory);
        container.setQueueNames(BOUTIQUE_QUEUE_NAME);
        container.setAcknowledgeMode(AcknowledgeMode.AUTO);
        return IntegrationFlows.from(Amqp.inboundAdapter(container)) /* Get Message from RabbitMQ */
                .handle(msg ->
                {
                    String msgString = new String((byte[]) msg.getPayload(), StandardCharsets.UTF_8);
                    HttpEntity<String> requestBody = new HttpEntity<String>(msgString, headers);
                    restTemplate.postForObject(ENDPOINT_LOCAL_URL, requestBody, String.class);
                    System.out.println(msgString);
                   
                })
                .get();
    }
EL HACHIMI
  • 17
  • 6

1 Answers1

0

For the first scenario, simply configure a inbound adapter for each and set the output channel to a common channel for the downstream flow.

For the second scenario, simply set the concurrentConsumers and maxConcurrentConsumers on the listener container and it will scale up/down the threads as needed.

See the Spring AMQP Documentation.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Thanks you Gary! I did set concurrentConsumers et 1 and maxConcurrentConsumers to 5. To test multithreading I added a sleep in the middle of the flow to block that thread and make spring use multiple threads. I remarked the program still uses only one thread (it waits until the message finish sleeping before taking the next message from the queue) it didn't create another thread. is this the way it's supposed to behave ? – EL HACHIMI Sep 18 '20 at 12:50
  • See the documentation I referred you to `>This works in conjunction with four additional properties: consecutiveActiveTrigger, startConsumerMinInterval, consecutiveIdleTrigger, and stopConsumerMinInterval. With the default settings, the algorithm to increase consumers works as follows:` `If the maxConcurrentConsumers has not been reached and an existing consumer is active for ten consecutive cycles AND at least 10 seconds has elapsed since the last consumer was started, a new consumer is started. A consumer is considered active if it received at least one message in batchSize * receiveTimeout` – Gary Russell Sep 18 '20 at 13:19