Stack Overflow
Stack Overflow

Questions tagged [apache-zookeeper]

Apache ZooKeeper is a distributed service that provides configuration information storage, naming, synchronization, coordination, and group services.

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. It was inspired by Chubby, Google's distributed lock system.

All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them, which makes them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

ZooKeeper is used by a variety of other software projects to maintain configuration information including Apache Storm, Apache Flink, Solr Cloud, Apache Kafka and Apache Hadoop.

Links

Related Tags

3686 questions
25
votes
1 answer

Best way to start Zookeeper automatically on Ubuntu Server 14.04?

I have installed Zookeeper using sudo apt-get install zookeeper on an Ubuntu server 14.04. I am having trouble understanding how to configure Zookeeper to start automatically, on boot. Also, I can manually start it successfully only with root…
user2916547
  • 1,963
  • 5
  • 17
  • 21
25
votes
4 answers

UnknownHostException kafka

I am trying to setup a Kafka cluster (the first node in the cluster actually). I have a single node zookeeper cluster setup. I am setting up kafka on a separate node. Both running CentOS 6.4, running IPV6 which is a bit of a PITA. I verified that…
noplay
  • 2,391
  • 6
  • 26
  • 33
24
votes
1 answer

Apache ZooKeeper: How do writes work

Apache ZooKeeper is a kind of high available data-store for small objects. A ZooKeeper cluster consists of some nodes which all keep the whole dataset in their memory. The dataset is called "always-consistent", so every node has the same data at…
theomega
  • 31,591
  • 21
  • 89
  • 127
24
votes
3 answers

Starting Kafka Server Permanently

I have setup Kafka on Amazon EC2 instance. I have done the following in below order: (1) SSH into the Instance (2) Start Zookeper (3) Start Kafka (4) Execute Producer and Consumer Programs. Everything is working fine till here. However once I close…
user1058797
  • 867
  • 3
  • 10
  • 19
23
votes
2 answers

Kafka SASL zookeeper authentication

I am facing the following error while enabling SASL on Zookeeper and broker authentication. [2017-04-18 15:54:10,476] DEBUG Size of client SASL token: 0 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-18 15:54:10,476] ERROR cnxn.saslServer…
sunder
  • 968
  • 2
  • 11
  • 31
23
votes
3 answers

Kafka-python retrieve the list of topics

I'm using kafka-python and I'm wondering if there is a way for showing all the topics. Something like this: ./bin/kafka-topics.sh --list --zookeeper localhost:2181
teoreda
  • 2,392
  • 1
  • 21
  • 28
23
votes
4 answers

What's the purpose of using Zookeeper rather than just databases for managing distributed systems?

I’m learning Zookeeper and so far I don't understand the purpose of using it for distributed systems that databases can't solve. The use cases I’ve read are implementing a lock, barrier, etc for distributed systems by having Zookeeper clients…
Glide
  • 20,235
  • 26
  • 86
  • 135
23
votes
16 answers

Apache Kafka error on windows - Couldnot find or load main class QuorumPeerMain

I just downloaded Kafka 2.8.0 from Apache website, and I am trying to setup using the instructions given on the website. But when I try to start zookeper server, I am getting below error: Error: Could not find or load main class…
Pradeep Simha
  • 17,683
  • 18
  • 56
  • 107
23
votes
3 answers

Concerns about zookeeper's lock-recipe

While reading the ZooKeeper's recipe for lock, I got confused. It seems that this recipe for distributed locks can not guarantee "any snapshot in time no two clients think they hold the same lock". But since ZooKeeper is so widely adopted, if there…
hulunbier
  • 233
  • 2
  • 5
22
votes
3 answers

Is ZooKeeper always consistent in terms of CAP theorem?

Is that correct that ZooKeeper is always CP (in terms of CAP theorem)? Or is there anyway to use it as AP for service discovery needs?
Artem
  • 7,275
  • 15
  • 57
  • 97
22
votes
5 answers

How do one read the Zookeeper transaction log?

Are there any existing tools that help to read the Zookeeper transaction log? By default, it is in binary format and I would like to read it in human readable form.
Sriram Subramanian
  • 2,623
  • 5
  • 32
  • 35
22
votes
1 answer

How do I check if a path exists in Zookeeper using Curator?

I'm currently in the early stages of development to integrate Zookeeper stuff into my app. I'm using Netflix's Curator as an API for dealing with Zookeeper. I get the gist of most of it (like adding ZNodes and whatnot). But how the heck do you check…
Cody S
  • 4,744
  • 8
  • 33
  • 64
21
votes
6 answers

How do I delete a Kafka Consumer Group to reset offsets?

I want to delete a Kakfa consumer group so that when the application creates a consumer and subscribes to a topic it can start at the beginning of the topic data. This is with a single node development vm using the current latest Confluent Platform…
clay
  • 18,138
  • 28
  • 107
  • 192
21
votes
2 answers

ZooKeeper keeps getting EndOfStreamException, causing a crash

My Zookeeper is controlling a few different queues for different jobs, by holding the relevant job data in each node until the computer is ready to process. If I stop the overall service, such that no jobs can be started ZooKeeper runs just fine…
Nixxon
  • 767
  • 1
  • 11
  • 24
21
votes
6 answers

Apache Curator Unimplemented Errors When Trying to Create zNodes

I'm attempting to use Apache Curator with a dockerized zookeeper instance and no matter how I attempt to connect I always end up with a org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for... error.…
snerd
  • 1,238
  • 1
  • 14
  • 28
1 2 3
…
99 100