0

Need to check what is the active controller from any of the log file(Control center or Kafka connect log file etc..). Actually we have multiple brokers out of that one is Active controller(for ex. Broker 01) and now it changed to Broker 02 (able to see from control center ui).

So we just want to when exactly it got changed (exact time) from Broker 01 to Broker 02. Please help me out where we can find this from log files or any other source when exactly it got changed.

Anilkumar Bathula
  • 278
  • 3
  • 18

1 Answers1

2

Have you restarted your broker01 instance? Controller is moving around and its normal behavior when the the current controller getting down another active broker is chosen to be the controller , in order to avoid two time of controller movement, the recommended way for rolling restart of kafka cluster is restarting the controller as the last broker getting restart in the cluster

By command cli you could check active controller from zookeeper command shell

./bin/zookeeper-shell.sh [ZK_IP] get /controller

You can check the broker's controller.log usually when the broker starts writing to controller.log its the time it is chosen as a controller , other brokers would not write to controller.log file at that time –

It could be your controller.log contains data from the previous time it was the controller , in your other cluster controller.log there should be DEBUG message if enabled states "broker x has been elected as the controller"

Ran Lupovich
  • 1,655
  • 1
  • 6
  • 13
  • Thanks @Ran Lupovich for the prompt response. We haven't done any restart and automatically it is changing to other broker as active controller. Ya i tried with above command on ZK Server. Am getting the just what is active controlelr and it's id. But I need exact time when it got changed to new active contoller. – Anilkumar Bathula Jun 03 '21 at 04:15
  • There should be controller.log on broker02 states the time when it got the "leadrship" , try checking it out – Ran Lupovich Jun 03 '21 at 04:23
  • Any particular string to check from the log file, I tried to check in control center log file. I am not sure which string that shows the new broker is elected as controller. – Anilkumar Bathula Jun 03 '21 at 04:25
  • Can you upload the controller.log from broker02? Not the control center log file, usually when the broker starts writing to controller.log its the time it is chosen as a controller , other brokers would not write to controller.log file at that time – Ran Lupovich Jun 03 '21 at 04:27
  • I see controller.log file even generated before the active controller changed from broker01 to broker02. In my case every 1 hr we have taken backup of that file. So not sure when it exactly with log file because I see manually active controller changed from c3 is on June2nd, but we have contoller.log files from last 10 back in log directory. – Anilkumar Bathula Jun 03 '21 at 07:29
  • 1
    It could be your controller.log contains data from the previous time it was the controller , in your other cluster controller.log there should be DEBUG message if enabled states "broker x has been elected as the controller" – Ran Lupovich Jun 03 '21 at 09:17