0

I am trying to set up a Zookeeper cluster for Pulsar. I am following the instructions here, but I keep failing.

In my setup, I have two nodes, that should be part of the cluster. Since I need to deploy bookie to the same nodes, I executed 

$ PULSAR_EXTRA_OPTS="-Dstats_server_port=8001" bin/pulsar-daemon start zookeeper

to start zookeeper. Afterwards, I am trying to init the cluster using this command:

bin/pulsar initialize-cluster-metadata \
  --cluster pulsar-cluster-1 \
  --zookeeper 10.100.100.77:2181 \
  --configuration-store 10.100.100.77:2181 \
  --web-service-url http://10.100.100.77:8080 \
  --broker-service-url pulsar://10.100.100.77:6650 \

But I keep getting this error: 

17:12:24.146 [main-SendThread(10.100.100.77:2181)] INFO  org.apache.zookeeper.ClientCnxn - Socket error occurred: 10.100.100.77/10.100.100.77:2181: Verbindungsaufbau abgelehnt
17:12:25.251 [main-SendThread(10.100.100.77:2181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.100.100.77/10.100.100.77:2181. Will not attempt to authenticate using SASL (unknown error)

I read here that I need to have an odd number of nodes, so I added a virtual machine on one of the nodes. When I start Zookeeper on it, it doesn't print an error message, but but shows:

$ PULSAR_EXTRA_OPTS="-Dstats_server_port=8001" bin/pulsar-daemon start zookeeper

doing start zookeeper ...
starting zookeeper, logging to /home/host1/apache-pulsar-2.4.0/logs/pulsar-zookeeper-host1-VirtualBox.log
OpenJDK 64-Bit Server VM warning: Option AggressiveOpts was deprecated in version 11.0 and will likely be removed in a future release.
[AppClassLoader@27c170f0] info AspectJ Weaver Version 1.9.2 built on Wednesday Oct 24, 2018 at 15:43:33 GMT
[AppClassLoader@27c170f0] info register classloader jdk.internal.loader.ClassLoaders$AppClassLoader@27c170f0
[AppClassLoader@27c170f0] info using configuration file:/home/host1/apache-pulsar-2.4.0/lib/org.apache.pulsar-pulsar-zookeeper-utils-2.4.0.jar!/META-INF/aop.xml
[AppClassLoader@27c170f0] info using configuration file:/home/host1/apache-pulsar-2.4.0/lib/org.apache.pulsar-pulsar-zookeeper-2.4.0.jar!/META-INF/aop.xml
[AppClassLoader@27c170f0] info register aspect org.apache.pulsar.zookeeper.SerializeUtilsAspect
[AppClassLoader@27c170f0] info register aspect org.apache.pulsar.broker.zookeeper.aspectj.ClientCnxnAspect

However the Zookeeper service is not started, even if the setup is very similar to its host and I can't make up why.

Any Ideas how I could proceed from here? Thanks in advance!

Evgeniy Berezovsky
  • 18,571
  • 13
  • 82
  • 156
coolaturon
  • 127
  • 2
  • 8

2 Answers2

0

The first error you posted seems to indicate that the connection to 10.100.100.77:2181 is refused "Verbindungsaufbau abgelehnt", and therefore the ZK server isn't listening on that server and port. You should first confirm that ZK is up and running and check the ZK log for any errors.

HTH

David Kjerrumgaard
  • 1,056
  • 7
  • 10
  • I thought so, too, but I was able to connect to it via telnet. However, the Zookeeper instance told me, that is was currently not processing any requests and ended the telnet session. This might have been caused by the instance being busy with the quorum. – coolaturon Sep 03 '19 at 10:10
  • FWIW, you can also achieve a quorum with one ZK instance as well. Perhaps you should try running a single node ZK ensemble instead. – David Kjerrumgaard Sep 04 '19 at 14:37
  • You are right. Today I noticed, that there were also some configuration mistakes involved, which led to the cluster not being deployed correctly. I really don't like that systems like pulsar are build up upon so many other, that each need to be deployed seperately and that are so vulnerable to misconfigurations, but anyway, everything should be running fine now. – coolaturon Sep 05 '19 at 20:52
0

I found the soulution. The original error was indeed caused by having an odd number of nodes. The third (virtual) one wouldn't start, because of a mislocation of Zookepers data-directory. Now that the third server started, also the configuration passed successfully.

coolaturon
  • 127
  • 2
  • 8