Confluent Control Center - Consumer is not listed

Question

I have the following code to connect to Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "myconfluentkafkabroker:9092");
props.put("group.id","test");
props.put("enable.auto.commit","true");
props.put("auto.commit.interval.ms","1000");
props.put(ConsumerConfig.GROUP_ID_CONFIG, "my_CG");
props.put("group.instance.id", "my_instance_CG_id");
props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
props.put("key.deserializer", Class.forName("org.apache.kafka.common.serialization.StringDeserializer"));
props.put("value.deserializer", Class.forName("org.apache.kafka.common.serialization.StringDeserializer"));

KafkaConsumer<String,String> consumer = new KafkaConsumer<String,String>(props);
consumer.subscribe(Arrays.asList("MyTopic"));
try {
    while (true) { 
        ConsumerRecords<String, String> records = consumer.poll(100);
        for (ConsumerRecord<String, String> record : records)
        {
            log.debug("topic = %s, partition = %d, offset = %d,"
                customer = %s, country = %s\n",
                record.topic(), record.partition(), record.offset(),
                record.key(), record.value());

            int updatedCount = 1;
            if (custCountryMap.countainsKey(record.value())) {
                updatedCount = custCountryMap.get(record.value()) + 1;
            }
            custCountryMap.put(record.value(), updatedCount)

            JSONObject json = new JSONObject(custCountryMap);
            System.out.println(json.toString(4));
        }
    }
} finally {
    consumer.close();
}

Code didn't throw any errors but I still don't see the consumer listed

would this be an issue?

props.put("group.instance.id", "my_instance_CG_id");

score 1 · Accepted Answer · answered Oct 25 '21 at 17:19

1

You should verify the information that you see with the built-in tools that Kafka provides like kafka-consumer-groups.sh

You'll also need to actually poll messages and commit offsets, not just subscribe before you will see anything.

Otherwise, for that specific Control Center dashboard, it may require you to add the Monitoring Interceptors into your client

answered Oct 25 '21 at 17:19

OneCricketeer

179,855
19
132
245

Yes, I do pull but it is printing 0 messages – One Developer Oct 25 '21 at 17:22
Well, then use CLI tools to verify there is zero lag – OneCricketeer Oct 25 '21 at 17:27
how to print the ConsumerConfig values? something like 2021-04-01T00:44:32.367+05:30 [APP/PROC/WEB/0] [OUT] 2021-03-31 19:14:32.367 INFO 38 --- [http-nio-8080-exec-9] org.apache.kafka.clients.consumer.ConsumerConfig : ConsumerConfig values: 2021-04-01T00:44:32.367+05:30 [APP/PROC/WEB/0] [OUT] allow.auto.create.topics = true 2021-04-01T00:44:32.367+05:30 [APP/PROC/WEB/0] [OUT] auto.commit.interval.ms = 5000 2021-04-01T00:44:32.367+05:30 [APP/PROC/WEB/0] [OUT] auto.offset.reset = latest 2021-04-01T00:44:32.367+05:30 [APP/PROC/WEB/0] [OUT] bootstrap.servers = [17xxx:9092] – One Developer Oct 25 '21 at 17:27
Those should automatically get printed if you properly setup a SLF4J logger – OneCricketeer Oct 25 '21 at 17:28
any sample links? it is not printing for me – One Developer Oct 25 '21 at 17:29
I usually use Logback, but Log4j2 would be better. This might help - https://www.baeldung.com/log4j2-appenders-layouts-filters – OneCricketeer Oct 25 '21 at 17:32
so, all I need is the Log4j2 setup. nothing specific to kafkaconsumer itself, correct? would assume console appender work as well? – One Developer Oct 25 '21 at 17:33
Correct. `kafka-clients` includes SLF4J. SLF4J requires an external logging library, such as log4j2 or logback – OneCricketeer Oct 25 '21 at 17:34
final question - props.put("group.instance.id", "my_instance_CG_id"); would this cause any issues? and prevent the consumer from reporting in the control center? – One Developer Oct 25 '21 at 17:37
1

Not that I know of. Remove it. Does it change anything? – OneCricketeer Oct 25 '21 at 17:38
What happens when you specify two out of three brokers in the app? Would it cause any issues? – One Developer Oct 25 '21 at 18:39
@KarthikeyanVijayakumar Nope. The bootstrap protocol would return all of them, anyway – OneCricketeer Oct 25 '21 at 18:53

Confluent Control Center - Consumer is not listed

1 Answers1