Kafka broker shut down even though the log dirs are present

Question

When I saw this error message:

ERROR Shutdown broker because all log dirs in /tmp/kafka-logs have failed (kafka.log.LogManager)

The first thought is "well the /tmp directory probably got cleared out by the O/S (linux) - so I should update the kafka config to point to something permanent. However the directory is present and has not been wiped:

ll /tmp/kafka-logs/
total 20
drwxrwxr-x  2 ec2-user ec2-user   38 Apr  7 16:56 __consumer_offsets-0
drwxrwxr-x  2 ec2-user ec2-user   38 Apr  7 16:56 __consumer_offsets-7
drwxrwxr-x  2 ec2-user ec2-user   38 Apr  7 16:56 __consumer_offsets-42
..
drwxrwxr-x  2 ec2-user ec2-user   38 Apr  7 16:56 __consumer_offsets-32
drwxrwxr-x  2 ec2-user ec2-user  141 Apr 12 02:49 flights_raw-0
drwxrwxr-x  2 ec2-user ec2-user  178 Apr 12 08:25 air2008-0
drwxrwxr-x  2 ec2-user ec2-user  141 Apr 12 13:38 testtopic-0
-rw-rw-r--  1 ec2-user ec2-user 1244 Apr 17 22:29 recovery-point-offset-checkpoint
-rw-rw-r--  1 ec2-user ec2-user    4 Apr 17 22:29 log-start-offset-checkpoint
-rw-rw-r--  1 ec2-user ec2-user 1248 Apr 17 22:30 replication-offset-checkpoint

So then what does this actually mean, why is it happening and what should be done to correct/avoid the error?

Please provide what rights for `kafka-logs` dir. Please specify what user runs the broker. — ADS, Apr 20 '19 at 10:20
The `/tmp/kafka-logs` is created automatically under/owned by the user running `kafka` which is `ec2-user` — WestCoastProjects, Apr 20 '19 at 11:42

score 0 · Accepted Answer · answered Apr 20 '19 at 12:14

0

In related question best answer suggests to delete log dir both for Kafka /tmp/kafka-logs and Zookeper /tmp/zookeeper.

Probably it's because of Kafka issue which was resolved in August 2018.

Hope this will help.

answered Apr 20 '19 at 12:14

ADS

708
3
14

My issue with this is then the _data_ is lost. – WestCoastProjects Apr 20 '19 at 21:00

Kafka broker shut down even though the log dirs are present

1 Answers1