Understanding spring cloud messaging with rabbitmq

Question

I think I have a problem understanding spring cloud messaging and can't find an answer to a "problem" I'm facing.

I have the following setup (using spring-boot 2.0.3.RELEASE).

application.yml

spring:
    rabbitmq:
      host: localhost
      port: 5672
      username: guest
      password: guest
      virtual-host: /
    cloud:
      stream:
        bindings:
          input:
            destination: foo
            group: fooGroup
          fooChannel:
            destination: foo

Service class

@Autowired
FoodOrderController foodOrderController;

@Bean
public CommandLineRunner runner() {
    return (String[] args) -> {
       IntStream.range(0,50).forEach(e -> foodOrderController.orderFood());
    };
}

@StreamListener(target = FoodOrderSource.INPUT)
public void processCheapMeals(String meal){
    System.out.println("This was a great meal!: "+ meal);
}

@StreamListener(target = FoodOrderSource.INPUT)
public void processCheapMeals1(String meal){
    System.out.println("This was a great meal!: "+ meal);
}

FoodOrderController

public class FoodOrderController {

    @Autowired
    FoodOrderSource foodOrderSource;

    public String orderFood(){
        var foodOrder = new FoodOrder();
        foodOrder.setCustomerAddress(UUID.randomUUID().toString());
        foodOrder.setOrderDescription(UUID.randomUUID().toString());
        foodOrder.setRestaurant("foo");
        foodOrderSource.foodOrders().send(MessageBuilder.withPayload(foodOrder).build());
       // System.out.println(foodOrder.toString());
        return "food ordered!";
    }
}

FoodOrderSource

public interface FoodOrderSource {
    String INPUT = "foo";
    String OUTPUT = "fooChannel";

    @Input("foo")
    SubscribableChannel foo();
    @Output("fooChannel")
    MessageChannel foodOrders();
}

FoodOrderPublisher

@EnableBinding(FoodOrderSource.class)
public class FoodOrderPublisher {
}

The setup is working, with the exception that both StreamListener receive the same messages. So everything get's logged twice. Reading the documentation, it says specifying a group inside the queues bindings, both the listeners will be registered inside the group and only one listener will receive a single message. I know that the example above is not sensible, but I want to mimic a multi-node environment with multiple listeners setup.

Why is the message received by both listeners? And how can I make sure that a message is only received once within a setup group?

According to the documentation, messages should also be auto-acknowledged by default, but I can't find anything that indicates that the messages actually get acknowledged. Am I missing something here?

Here's some screenshots of rabbit admin

score 2 · Accepted Answer · answered Aug 18 '18 at 13:01

2

Reading the documentation, it says specifying a group inside the queues bindings, both the listeners will be registered inside the group and only one listener will receive a single message.

That is true when the listeners are in different application instances. When there are multiple listeners in the same instance they all get the same message. This is typically used with a condition where each listener can express interest in which meals they are interested in. Documented here.

Basically, the competing consumer is the binding itself which dispatches the message to the actual @StreamListeners in the application.

So, you can't "mimic a multi-node environment with multiple listeners setup" this way.

but I can't find anything that indicates that the messages actually get acknowledged

What do you mean by that? If the message is processed successfully, the container acks the message and it is removed from the queue.

answered Aug 18 '18 at 13:01

Gary Russell

166,535
14
146
179

Thanks! I've sinced created another spring boot app, set it up like the first instance, started them both and again, all messages are logged in both instances. Can you confirm that my configuration made in application.yml is correct? – baao Aug 18 '18 at 15:15
It looks ok; did you add the `ApplicationRunner` to both instances too? That would double up the messages. You can look on the RabbitMQ Admin UI (`http://localhost:15672`) to see the queue `foo.fooGroup` and its consumers. – Gary Russell Aug 18 '18 at 15:22
I've added some pictures, I don't see a connection being created between the group and the queues, do you? foo exchange seems to have both the queues, but I don't get the connection to the group. I did not add the ApplicationRunner to the second app, that's running only in the first one. – baao Aug 18 '18 at 15:32
Strange - you have 2 anonymous queues, which means the `group` property is not being applied for some reason. The queues are properly bound to exchange `foo`; there should not be an exchange `fooGroup`. There should be a single queue `foo.fooGroup` to which both consumers are attached, with the queue bound to `foo`. If you can't figure it out, post your project someplace like github; I can take a look to see what's wrong. But I'll only be around for a short time today so it might be tonight or tomorrow (EDT). – Gary Russell Aug 18 '18 at 15:44
I've just changed the StreamListener from using `target = FoodOrderSource.INPUT` to `Sink.INPUT`. That distributes the messages uniquely to both and creates the queues like you've described, but throws errors that application-1.foo has no subscribers. I'll try to figure it out; if I can't I'll upload to github and ping you. Thanks for that offer! – baao Aug 18 '18 at 15:51
1

Ah - it's due to this `@Input("foo")` - you need `bindings: foo:` instead of `bindings: input:`. Sorry I didn't notice the mismatch in the binding name. If there is no explicit binding for `foo`, it uses the defaults (destination = `foo`, anonymous consumer). – Gary Russell Aug 18 '18 at 15:56

score 1 · Answer 2 · answered Aug 18 '18 at 18:41

1

Thow correct answer is already replied on the post, but you can still look into this:

https://github.com/jinternals/spring-cloud-stream

answered Aug 18 '18 at 18:41

Mradul Pandey

2,034
1
14
12

Thanks, that looks helpful and will solve my next challenge! – baao Aug 18 '18 at 18:45
please like repo if you find it helpful, happy coding :) – Mradul Pandey Aug 18 '18 at 18:47

Understanding spring cloud messaging with rabbitmq

2 Answers2