7

I'm just starting understanding and trying ZeroMQ.

It's not clear to me how could I have a two way communication between more than two actors (publisher and subscriber) so that each component is able both to read and write on the MQ.

This would allow to create event-driven architecture, because each component could be listening for an event and reply with another event.

Is there a way to do this with ZeroMQ directly or I should implement my own solution on top of that?

user1978591
  • 229
  • 1
  • 5
  • 14

1 Answers1

10

If you want simple two-way communication then you simply set up a publishing socket on each node, and let each connect to the other.

In an many to many setup this quickly becomes tricky to handle. Basically, it sounds like you want some kind of central node that all nodes can "connect" to, receive messages from and, if some conditions on the subscriber are met, send messages to.

Since ZeroMq is a simple "power-socket", and not a message queue (hence its name, ZeroMQ - Zero Message Queue) this is not feasible out-of-the-box.

A simple alternative could be to let each node set up an UDP broadcast socket (not using ZeroMq, just regular sockets). All nodes can listen in to whatever takes place and "publish" its own messages back on the socket, effectively sending it to any nodes listening. This setup works on a LAN and in a setting where it is ok for messages to get lost (like periodical state updates). If the messages needs to be reliable (and possibly durable) you need a more advanced full-blown message queue.

If you can do without durable message queues, you can create a solution based on a central node, a central message handler, to which all nodes can subscribe to and send data to. Basically, create a "server" with one REP (Response) socket (for incoming data) and one PUB (Publisher) socket (for outgoing data). Each client then publishes data to the servers REP socket over a REQ (Request) socket and sets up a SUB (Subscriber) socket to the servers PUB socket.

Check out the ZeroMq guide regarding the various message patterns available.

To spice it up a bit, you could add event "topics", including server side filtering, by splitting up the outgoing messages (on the servers PUB socket) into two message parts (see multi-part messages) where the first part specifies the "topic" and the second part contains the payload (e.g. temp|46.2, speed|134). This way, each client can register its interest in any topic (or all) and let the server filter out only matching messages. See this example for details.

Basically, ZeroMq is "just" an abstraction over regular sockets, providing a couple of messaging patterns to build your solution on top of. However, it relieves you of a lot of tedious work and provides scalability and performance out of the ordinary. It takes some getting used to though. Check out the ZeroMq Guide for more details.

Jakob Möllås
  • 4,239
  • 3
  • 33
  • 61
  • By reading some resources it seems to me that the best way to do this the way I mean would be create a publishing socket on each node, as you said. Will that mean that I will need to explicitly send messages to all this sockets? I'm thinking of something like a MQ similiar to a cloud, where every component can just be listening on and write to, allowing me to add-remove components without the need to modify each other. Don't know if i'm clear on what I'd like to do. – user1978591 Feb 06 '13 at 17:10
  • Yes, all subscribers need to subscribe to all other nodes publishing sockets, that is why I do not think that is a feasible solution. As I mentioned in my answer, another way would be to create your own such MQ service, or messaging server, as you mention, based on ZeroMq sockets. That way nodes would not have to subscribe to every other nodes sockets, just the publishing socket of this server. It should not be too hard to create this using ZeroMq. The alternative is otherwise to set up something ready-made, like RabbitMQ. – Jakob Möllås Feb 06 '13 at 18:01
  • Yeah I think I'll try building it. Many thanks, you were very kind. – user1978591 Feb 06 '13 at 18:10
  • 1
    Are you sure this information is accurate? The "Zero" in ZeroMQ does not mean "no message queue" (See the [Zen of ZeroMQ](http://zguide.zeromq.org/page:all#The-Zen-of-Zero)). It does provide durable message queues. Also, it does come with various devices allowing for X, such as XPUB, which allow you to connect socket types back to back. These devices can either use the default logic to simply route messages, or can do custom code in the middle:[api](http://api.zeromq.org/2-1:zmq-device), [espresso pattern](http://zguide.zeromq.org/page:all#Pub-Sub-Tracing-Espresso-Pattern)) – jdi Apr 30 '13 at 08:36
  • @jdi Thanks, now I suddenly wonder where I came up with my information regarding the naming :-) However, it does no longer support durable queues (I assume you mean ZMQ_SWAP?). I think it was removed in version 3. At least it is not there in 3.2 (latest stable version). – Jakob Möllås May 01 '13 at 17:54
  • 3
    ZMQ provides a device that simply forwards messages from one socket (probably a 'sub' sock) to another (probably a 'pub' sock). For 2-way async comms, just give each actor a pub and sub sock, then connect the actors' pub socks to the forwarder's sub sock (and vice versa). Use `bind` in the forwarder device, and `connect` in the actors. Now, you have a single connection point for all of your actors. – Jacob Robbins May 02 '13 at 13:56
  • @JacobRobbins You are 100% correct, ZeroMq supports N-to-N pub/sub with ZMQ proxy, shame this is not the accepted answer. – raffian May 15 '13 at 18:43