Zookeeper CLI failing - IOException Packet is out of range

Question

Running zookeeper 3.3.3. I have a znode that I am just trying to list, via the CLI, as in:

ls /myznode/subznode

This crashes with an IOException in org.apache.ClientCnxn$SendThread.readLength at line 710.

Anyone seen this?? Someone suggested that maybe bad data is in the znode. Not sure if, or how... but I cannot delete it either, as it has something in it.

score 12 · Answer 1 · answered Nov 14 '13 at 23:31

12

I was able to work around this by increasing the max size of my listing call.

I added the "-Djute.maxbuffer=50111000" to my zkCli.sh script so that it started the client using the following line:

$JAVA "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
 "-Djute.maxbuffer=49107800" -cp "$CLASSPATH" $CLIENT_JVMFLAGS $JVMFLAGS \
 org.apache.zookeeper.ZooKeeperMain "$@"

I was then able to list & use rmr /big/node

answered Nov 14 '13 at 23:31

Charles L.

5,795
10
40
60

Fantastic, thanks! Even easier, from the shell run the script like `JVMFLAGS="$JVMFLAGS -Djute.maxbuffer=49107800 zkCli.sh ...`, no editing required. – Felix Frank Jan 18 '19 at 14:38

score 4 · Accepted Answer · answered May 24 '12 at 14:04

so, the problem was that the znode in question has been overwhelmed with sub-znodes. It had about 5 million of them. Zookeeper apparently does not like this. Even worse, there is no great way to clean it up. ZK should have a prune command (or something). Thanks for the answers.

Luca Geretti · Answer 3 · 2012-04-27T14:52:11.737

Given the error line you mentioned,

707         void readLength() throws IOException {
708             int len = incomingBuffer.getInt();
709             if (len < 0 || len >= packetLen) {
710                 throw new IOException("Packet len" + len + " is out of range!");
711             }
712             incomingBuffer = ByteBuffer.allocate(len);
713         }

it may be that your packet length is larger than what jute.maxBuffer allows. The default value reads 4M, and that should suffice, but you may have defined the property with a sensibly lower value. In any case, do you have a very large number of children?

score 3 · Answer 4 · answered Feb 01 '17 at 17:33

I had the same Packet out of range exception, but for a much silly reason. The port number I was specifying was for my solr instance and not the embedded zookeeper. I updated that to

bin/solr zk upconfig -z http://localhost:9983/ -n mynewconfig -d /path/to/configset where my instance is running on 8983.

Embedded zookeeper port defaults to localhost:(hostPort+1000)

Hope it helps anyone who is starting out like me.

score 1 · Answer 5 · answered May 25 '15 at 08:10

1

In my case, this error was due to a "bugged" version of SolrCloud (4.8.0), upgrading to latest (4.8.1) the problem disappeared.

answered May 25 '15 at 08:10

freedev

25,946
8
108
125

score 1 · Answer 6 · answered Feb 20 '18 at 16:35

1

I had a similar problem and I was able to fix it by fixing the zookeeper ports to 2181

answered Feb 20 '18 at 16:35

CMPE

1,853
4
21
37

score 0 · Answer 7 · answered Apr 24 '12 at 20:47

Have you tried to access it programmatically? Something like

ZooKeeper zooKeeper = new ZooKeeper(hostPort, 3000, myWatcher);
String path = "/myznode/subznode"
List<String> children = zooKeeper.getChildren(path, false);
for (String child : children) {
    System.out.println(child);
}

Zookeeper CLI failing - IOException Packet is out of range

7 Answers7

Linked