0

I have few doubts on how producer consumer recipe will work with Zookeeper.

All the examples I have seen so far have producers and consumers in the same java processes as separate thread. But the whole point of being distributed is to have different instances ( multiple ) of producer and consumer.

having different instances of producers and consumer means no local locks ( syntonization ) will be of any use.

Also because multiple processes are watching the same root queue node, they all will start processing a particular incoming entry.

What I am getting wrong here ???

Gaurav Singh
  • 47
  • 7
  • It is hard to tell what you are asking here. Maybe point to a specific example/recipe, and ask questions about that. Zookeeper works (indeed was designed for) multiple processes. – sbridges May 14 '13 at 03:41
  • @sbridges Thank you and appologies if I was ambiguous there. As we see in Java Linked Blocking Queue, take() method takes lock/gaurds the tail node before dequeueing from queue. So no 2 threads will take the same node from the tail. In case of zookeeper, all the process watching the root node of queue will get notified and all of them can grab the same set of nodes and start processing the same nodes. – Gaurav Singh May 15 '13 at 06:22
  • @sbridges So how can I call this a distributed queue then. I am talking of the producer consumer reciepe of Zookeeper. http://zookeeper.apache.org/doc/r3.4.1/zookeeperTutorial.html#sc_producerConsumerQueues – Gaurav Singh May 15 '13 at 06:24

1 Answers1

0

FYI - to see an example of producer/consumer in ZooKeeper have a look at Curator (disclaimer: I'm the author of Curator): http://curator.incubator.apache.org/curator-recipes/distributed-queue.html

NOTE: It's my opinion that ZooKeeper is a bad choice for queues. Here's a post that explains why: https://cwiki.apache.org/confluence/display/CURATOR/TN4

Randgalt
  • 2,907
  • 1
  • 17
  • 31