I have one interesting scenario. Seems like when there are no new topics to pick up (at least that's what I think is happening), my consumer suddenly shuts down.
I am using Kotlin + Spring Boot Kafka Producer and Consumer. My consumer is configured like this:
spring:
profiles:
active: ${SPRING_PROFILE:prod}
cloud:
stream:
bindingRetryInterval: ${BINDING_RETRY_INTERVAL:0}
default:
contentType: application/*+avro
group: my-app
consumer:
useNativeDecoding: false
concurrency: ${CONSUMER_CONCURRENCY:3}
maxAttempts: ${CONSUMER_MAX_ATTEMPTS:3}
bindings:
outbox:
# Topic to consume from
destination: my_outbox_topic
kafka:
bindings:
outbox:
consumer:
enableDlq: ${ENABLE_DLQ:false}
binder:
brokers: ${BOOTSTRAP_SERVERS:localhost:9092}
configuration.ssl.endpoint.identification.algorithm: ${SSL_ALGORITHM:}
consumerProperties:
schema.registry.url: ${SCHEMA_REGISTRY_URL:http://localhost:8081}
key.deserializer: org.apache.kafka.common.serialization.StringDeserializer
value.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
specific.avro.reader: false
So I start the app, it consumes for a while and then I get these types of logs:
2021-09-28 12:57:38.113 INFO 4254 --- [container-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-2,
groupId=youscript-app] Member consumer-2-dc0e4115-6e9a-4a99-9d04-a3a2efb2c7b3 sending LeaveGroup request to coordinator __MASKES__:9094 (id: 2147483646 rack: null)
2021-09-28 12:57:38.113 INFO 4254 --- [container-0-C-1] o.a.k.clients.consumer.KafkaConsumer : [Consumer clientId=consumer-2, groupId=my-app] Unsubscribed all topics or patterns and assigned partitions
2021-09-28 12:57:38.113 INFO 4254 --- [container-0-C-1] o.s.s.c.ThreadPoolTaskScheduler : Shutting down ExecutorService
2021-09-28 12:57:38.120 INFO 4254 --- [container-1-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-3, groupId=my-app] Member consumer-3-701431a3-7cb9-4962-8f89-f2df41a176fd sending LeaveGroup request to coordinator __MASKED__:9094 (id: 2147483646 rack: null)
2021-09-28 12:57:38.120 INFO 4254 --- [container-1-C-1] o.a.k.clients.consumer.KafkaConsumer : [Consumer clientId=consumer-3, groupId=my-app] Unsubscribed all topics or patterns and assigned partitions
2021-09-28 12:57:38.120 INFO 4254 --- [container-1-C-1] o.s.s.c.ThreadPoolTaskScheduler : Shutting down ExecutorService
2021-09-28 12:57:38.189 INFO 4254 --- [container-1-C-1] essageListenerContainer$ListenerConsumer : my-app: Consumer stopped
2021-09-28 12:57:38.197 INFO 4254 --- [container-0-C-1] essageListenerContainer$ListenerConsumer : my-app: Consumer stopped