How do one read the Zookeeper transaction log?

Question

Are there any existing tools that help to read the Zookeeper transaction log? By default, it is in binary format and I would like to read it in human readable form.

score 37 · Answer 1 · edited Jun 24 '23 at 05:19

37

I don't know if you have solved this question.

cd /your/zookeeper/directory/

If you want to read snapshots, use:

java -cp zookeeper-3.4.6.jar:lib/log4j-1.2.16.jar:lib/slf4j-log4j12-1.6.1.jar:lib/slf4j-api-1.6.1.jar org.apache.zookeeper.server.SnapshotFormatter version-2/snapshot.xxx

If you want to read logs, use:

java -cp zookeeper-3.4.6.jar:lib/log4j-1.2.16.jar:lib/slf4j-log4j12-1.6.1.jar:lib/slf4j-api-1.6.1.jar org.apache.zookeeper.server.LogFormatter version-2/log.xxx

edited Jun 24 '23 at 05:19

Johann8

649
7
20

answered Nov 25 '15 at 07:24

Oliver

371
3
4

4 years later and your post is still helpful! Thanks a bunch Oliver and Tony. I was attempting to run this command on the server log inside of the logs folder (not realizing I was meant to run it on the transaction logs...) – plum 0 Nov 25 '19 at 09:55
with zookeeper 3.5.6 this worked for me (updated jar names and had to add jute): `java -cp lib/zookeeper-3.5.6.jar:lib/log4j-1.2.17.jar:lib/slf4j-log4j12-1.7.25.jar:lib/slf4j-api-1.7.25.jar:lib/zookeeper-jute-3.5.6.jar` – xref May 15 '20 at 18:03
The LogFormatter tool is removed in 3.7 – filimonov Nov 30 '20 at 09:21

score 7 · Answer 2 · edited Mar 23 '15 at 20:59

7

You can use something like this

java -cp $ZOOKEEPER_CLASSPATH org.apache.zookeeper.server.LogFormatter [zookeeper log file path]

edited Mar 23 '15 at 20:59

Conspicuous Compiler

6,403
1
40
52

answered Oct 09 '13 at 04:41

Vikas Vishwakarma

71
1
2

1

I can't make it work. Keeps returning `Error: Could not find or load main class log.5200768f38` – esengineer Oct 20 '14 at 08:07
@Eye - You need a valid value for `ZOOKEEPER_CLASSPATH` set before you can execute Vikas's command. I personally prefer to simply reuse HBase's classpath setup: `hbase org.apache.zookeeper.server.LogFormatter ` which automatically loads the class. – Harsh J May 20 '16 at 03:50

score 1 · Answer 3 · answered May 15 '20 at 18:17

Building on the previous two answers, using Zookeeper 3.5.6: from the /path/to/zookeeper/lib dir which contains all the ZK and supporting jars run:

java -cp * org.apache.zookeeper.server.LogFormatter /path/to/zookeeper/transaction/logs/version-2/log.xxx

score 1 · Answer 4 · answered Nov 30 '20 at 09:24

Since zookeeper version 3.6 there are tools to read transaction log and snapshots in zookeeper distribution:

For transaction log:

bin/zkTxnLogToolkit.sh --dump /datalog/version-2/log.f3aa6

For snapshots:

./zkSnapShotToolkit.sh -d /data/zkdata/version-2/snapshot.fa01000186d

See the details in official docs

score 0 · Answer 5 · answered Jun 20 '20 at 15:16

You can enable ZooKeeper audit logs for ZooKeeper 3.6.o and upwards. To enable audit logs configure audit.enable=true in conf/zoo.cfg.

One thing to keep in mind is that logs from different servers of the same ensemble should be aggregated because each one of them includes operations that where executed only form clients connected to this particular server.

Full information here: https://zookeeper.apache.org/doc/r3.6.1/zookeeperAuditLogs.html

How do one read the Zookeeper transaction log?

5 Answers5