0

I have the following simple piece of code:

    private int i = 0;

    @StreamListener(Sink.INPUT)
    public void processMessage(Message<?> message) {
         i++;
    }

    @Scheduled(fixedDelay=5000)
    private void scheduled(){
        LOG.info("Messages consumed: " + i);
    }

and the following properties:

spring.cloud.stream.bindings.input.consumer.headerMode=raw
spring.cloud.stream.kafka.binder.autoCreateTopics=false
spring.cloud.stream.kafka.bindings.input.consumer.autoCommitOffset=false
spring.cloud.stream.bindings.input.destination=test6
spring.cloud.stream.bindings.input.group=testGroup50
spring.cloud.stream.bindings.input.partitioned=false

I have a local kafka topic with a single partition with 96 k messages. The simple kafka consumer provided by the kafka library consumes those messages in roughly 4 seconds.

However, the above code takes close to 1 minute!

enter image description here

Obviously, it is a concern for our application, has anyone experienced this before? Am I missing something here?

Visual VM isn't flagging anything either.

PS: I just tried with auto commit and I am still seeing an atrocious performance.

ltalhouarne
  • 4,586
  • 2
  • 22
  • 31

2 Answers2

1

I think it would be best if you opened a Git Hub issue in https://github.com/spring-cloud/spring-cloud-stream-binder-kafka.

Also, can you indicate what version are you using and what is your message size, please, so that we can repeat the measurements? Thanks.

Edit: As per the discussion in https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/71, this seems to be specific to macOS (so far reproduced on Sierra and El Capitan) and the 0.9 client)

Marius Bogoevici
  • 2,380
  • 1
  • 13
  • 14
1

Thanks to @Marius Bogoevici's research, it was discovered that the poor performance was caused by Mac OS:

https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/71#issuecomment-263280685

Posting this for reference in case anyone else encounters this issue.

ltalhouarne
  • 4,586
  • 2
  • 22
  • 31