Stack Overflow
Stack Overflow

Questions tagged [apache-zookeeper]

Apache ZooKeeper is a distributed service that provides configuration information storage, naming, synchronization, coordination, and group services.

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. It was inspired by Chubby, Google's distributed lock system.

All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them, which makes them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

ZooKeeper is used by a variety of other software projects to maintain configuration information including Apache Storm, Apache Flink, Solr Cloud, Apache Kafka and Apache Hadoop.

Links

Related Tags

3686 questions
34
votes
17 answers

Kafka Broker doesn't find cluster id and creates new one after docker restart

I've created docker with kafka broker and zookeeper to start it with run script. If I do fresh start it starts normally and runs ok (Windows -> WSL -> two tmux windows, one session). If I shut down kafka or zookeeper and start it again it will…
Bohdan Myslyvchuk
  • 1,657
  • 3
  • 24
  • 39
31
votes
6 answers

Why does zookeeper not use my log4j.properties file log directory

In my zookeeper/conf/log4j.properties file I set the zookeeper.log.dir to $HOME/zklogs When I use zkServer.sh it does not use that directory. Instead it uses the ${ZOO_LOG_DIR} which when I echo it, comes out to "." I don't understand how fix this…
smuggledPancakes
  • 9,881
  • 20
  • 74
  • 113
31
votes
2 answers

ZooKeeper reliability - three versus five nodes

From the ZooKeeper FAQ: Reliability: A single ZooKeeper server (standalone) is essentially a coordinator with no reliability (a single serving node failure brings down the ZK service). A 3 server ensemble (you need to jump to 3 and not 2 because…
user650654
  • 5,630
  • 3
  • 41
  • 44
31
votes
5 answers

zookeeper not starting

i'm working with zookeeper (http://zookeeper.apache.org/). downloaded 3.3.5 and create zoo.cfg and placed in $ZOOKEEPER/conf, started the zookeeper using zkServer start. but following is the error can any please help me here .. nfig or no quorum…
Krishna Gangaraju
  • 519
  • 1
  • 6
  • 12
29
votes
9 answers

Classpath is empty. Please build the project first e.g. by running './gradlew jar -PscalaVersion=2.11.12'

I am not able to run a Apache Kafka service due to a failure while trying to start a Zookeeper instance. I have downloaded and tried it with all 3 availabe downloads at the official site. (binarys and source) When i try to start zookeeper with…
Andreas bleYel
  • 463
  • 2
  • 5
  • 7
29
votes
2 answers

Zookeeper vs In-memory-data-grid vs Redis

I've found different zookeeper definitions across multiple resources. Maybe some of them are taken out of context, but look at them pls: A canonical example of Zookeeper usage is distributed-memory computation... ZooKeeper is an open source…
VB_
  • 45,112
  • 42
  • 145
  • 293
28
votes
7 answers

Starting Zookeeper Cluster. Error: Could not find or load main class org.apache.zookeeper.server.quorum.QuorumPeerMain

(I'm running on CentOS 5.8). I've been following the direction for a Clustered (Multiserver) Zookeeper Set-up, but getting an error when I try to start up my server. When I run the command as described in the documentation: java -cp…
Shark
  • 2,322
  • 5
  • 31
  • 44
27
votes
8 answers

Is there any way to configure nginx (or other quick reverse proxy) dynamically?

Suppose we have several identical nodes which are the application servers of some n-tier service. And suppose we use Apache ZooKeeper to keep all the config's of our distributed application. Plus we have an nginx as a load balancer and reverse proxy…
Dima
  • 547
  • 1
  • 4
  • 16
27
votes
1 answer

Why is kafka not creating a topic? bootstrap-server is not a recognized option

I am new to Kafka and trying to create a new topic on my local machine. I am following this link. Here are the steps which i followed: Start zookeeper bin/zookeeper-server-start.sh config/zookeeper.properties Start…
KayV
  • 12,987
  • 11
  • 98
  • 148
27
votes
13 answers

error while starting kafka broker

I was able to successfully set up zookeeper and one kafka broker yesterday. Everything worked as expected. I shut down kafka (ctrl + c) and then zookeeper. Today I started zookeeper and when I started kafka (bin/kafka-server-start.sh…
brain storm
  • 30,124
  • 69
  • 225
  • 393
27
votes
5 answers

Kafka: Get broker host from ZooKeeper

For particular reasons I need to use both - ConsumerGroup (a.k.a. high-level consumer) and SimpleConsumer (a.k.a. low-level consumer) to read from Kafka. For ConsumerGroup I use ZooKeeper-based config and am completely satisfied with it, but…
ffriend
  • 27,562
  • 13
  • 91
  • 132
27
votes
3 answers

Kafka - How to commit offset after every message using High-Level consumer?

I'm using Kafka's high-level consumer. Because I'm using Kafka as a 'queue of transactions' for my application, I need to make absolutely sure I don't miss or re-read any messages. I have 2 questions regarding this: How do I commit the offset to…
Hongyi Li
  • 1,059
  • 2
  • 11
  • 19
27
votes
1 answer

what is zookeeper port and its usage?

I am quite new for zookeeper port through which I am coming across from past few days. I introduced with zookeeper port keyword at two occasion: while configuring neo4j db cluster (link) and while running compiled voltdb catalog (link) (See Network…
Aman Gupta
  • 5,548
  • 10
  • 52
  • 88
26
votes
1 answer

Got user-level KeeperException when processing

I know this has been asked before, so please forgive me for asking again, but none of the solutions pointed to by that thread worked for me. I'm trying to setup Kafka 0.10.20 following the docs. When I start ZooKeeper, I get the following logs: $…
B. Banner
  • 261
  • 1
  • 3
  • 4
25
votes
3 answers

How can I delete a data node which is not empty in zookeeper?

For deleting a node in zookeeper from command line interface there is a delete command. When I run it, it says: Node not empty: /testNode I am using this guide.
Amit Kumar
  • 2,685
  • 2
  • 37
  • 72
1 2
3
…
99 100