0

I have a kafka consumer (written in python). It listens to topics with regex(^.*_collection_updates$).

I have seen that the connection between consumer and brokers stops and starts automatically. I don't know the reason. Can someone tell me diff between below stack overlflow listed configuration ?

The answer in this issue is the configuration i wanted to know.

JR ibkr
  • 869
  • 7
  • 24
Rangeesh
  • 361
  • 1
  • 13
  • 1
    How many partitions you have ? How many consumer groups? How many consumers ? Which version of Kafka? – JR ibkr Jun 24 '19 at 17:35
  • partition - 1 , consumer_groups - 4 , consumers - 4 , kafka_version - confluent-kafka==0.11.6 – Rangeesh Jun 25 '19 at 10:59
  • Did you try 'auto.offset.reset': 'earliest' ? Default value of this parameter is latest https://kafka.apache.org/documentation/#consumerconfigs. – JR ibkr Jun 25 '19 at 14:48
  • What is your question exactly ? – JR ibkr Jun 25 '19 at 14:55
  • Just want to know , difference between first and second below. 1.The problem seems to have been in the Consumer() arguments. This doesn't work properly: self.kafka_consumer = confluent_kafka.Consumer({ 'group.id': 'mygroup', 'bootstrap.servers': 'kafka:9092', 'auto.offset.reset': 'earliest', }) 2.But this does: self.kafka_consumer = confluent_kafka.Consumer({ 'group.id': 'mygroup', 'bootstrap.servers': 'kafka:9092', 'default.topic.config': { 'auto.offset.reset': 'earliest', }, }) – Rangeesh Jun 26 '19 at 06:27
  • That is too vague. What kind of error are you getting in 1st case? Have you checked kafka document? Is default.topic.config a parameter that consumer accepts? – JR ibkr Jun 26 '19 at 13:35
  • I am marking this question vague. – JR ibkr Jun 26 '19 at 13:36

0 Answers0