0

scheme: 2 broker -2 hosts Java clients - org.apache.kafka.clients.producer.KafkaProducer or org.apache.kafka.clients.consumer.KafkaConsumer Bootsrap servers property contain 2 ip address (2 brokers) Zookeeper - distributed coordinator

Invocation Method at constructor org.apache.kafka.clients.ClientUtil#parseAndValidateAddresses check only network up servers without validate instance (check broken state server or doesn't start)

shkiper
  • 111
  • 4
  • Doesn't the Kafka client keep trying to reconnect automatically? – Thilo May 07 '18 at 11:51
  • No, it doesn't. Get first address and throw exception – shkiper May 07 '18 at 12:55
  • Could be address is not reachable. Provide all brokers address if possible to avoid downtime of application. – daemonsl May 08 '18 at 08:57
  • Thank you) Yes, i used library zookeeper for java. stackoverflow.com/questions/29490113/… for list brokers, but implemented watcher stub ( new ZooKeeper("localhost:2181", 10000, new Watcher() - no null parameter, because throw NPE after fetch data); – shkiper May 10 '18 at 12:04

1 Answers1

0

By default, Zookeeper keep tracks of all nodes connected and reachable brokers. You can query ZK to see what nodes are connected at present.

# Connect to ZooKeeper
bin/zookeeper-shell.sh  localhost:2181
Connecting to localhost:2181
Welcome to ZooKeeper!
JLine support is disabled

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
    >> ls /brokers/ids
        [1, 2, 3]
    >> ls /brokers/topics
        [myTopic, __consumer_offsets]
    >> get /brokers/ids/0
    Node does not exist: /brokers/ids/0
    >> get /brokers/ids/1
    {"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT","SSL":"SSL"},"endpoints":["PLAINTEXT://172.31.4.17:9092","SSL://18.216.252.106:9093"],"jmx_port":9999,"host":"172.31.4.17","timestamp":"1522349056258","port":9092,"version":4}
    cZxid = 0x3000005f3
    ctime = Thu Mar 29 18:44:16 UTC 2018
    mZxid = 0x3000005f3
    mtime = Thu Mar 29 18:44:16 UTC 2018
    pZxid = 0x3000005f3
    cversion = 0
    dataVersion = 0
    aclVersion = 0
    ephemeralOwner = 0x2625d127c0b0003
    dataLength = 234
    numChildren = 0
daemonsl
  • 442
  • 4
  • 6
  • Thank you) Yes, i used library zookeeper for java. https://stackoverflow.com/questions/29490113/kafka-get-broker-host-from-zookeeper for list brokers, but implemented watcher stub ( new ZooKeeper("localhost:2181", 10000, new Watcher() - no null parameter, because throw NPE after fetch data); – shkiper May 10 '18 at 12:04