12

I have to develop a message bus for processes to send, receive messages from each other. Currently, we are running on Linux with the view of porting to other platforms later.

For this, I am using ZeroMQ over TCP. The pattern is PUB-SUB with a forwarder. My bus runs as a separate process and all clients connect to SUB port to receive messages and PUB to send messages. Each process subscribes to messages by a unique tag. A send call from a process sends messages to all. A receive call will fetch that process the messages marked with the tag of that process. This is working fine.

Now I need to wrap the ZeroMQ stuff. My clients only need to supply a unique tag. I need to maintain a global list of tags vs. ZeroMQ context and sockets details. When a client say, initialize_comms("name"); the bus needs to check if this name is unique, create ZeroMQ contexts and sockets. Similarly, if a client say receive("name"); the bus needs to fetch messages with that tag.

To summarize the problems I am facing;

  1. Is there anyway to achieve this using facilities provided by ZeroMQ?
  2. Is ZeroMQ the right tool for this, or should I look for something like nanomsg?
  3. Is PUB-SUB with forwarder the right pattern for this?
  4. Or, am I missing something here?
marsop
  • 323
  • 4
  • 19
fortytwo
  • 491
  • 1
  • 5
  • 16
  • You could check out the [OMG DDS](http://en.wikipedia.org/wiki/Data_Distribution_Service) (Data Distribution Service) standard. From your (brief) description, it seems like a good fit and provides the kind of abstraction that you are looking for. – Reinier Torenbeek Jul 07 '14 at 23:48
  • I'm not entirely clear on why what you're looking to do isn't completely covered under normal subscriptions in ZMQ. – Jason Jul 08 '14 at 13:58
  • @ReinierTorenbeek > Thank you for the pointers. A quick glance and the theory is perfect for what I have in mind but it seems as if there aren't that many open source implementations. I'll keep an eye. – fortytwo Jul 08 '14 at 22:50
  • @Jason > I am actually quite new to ZeroMQ so you may be right. Why I chose PUB-SUB with FORWARDER is that I can't make any of the processes static. They can come and go as they like so I needed a static point of contact and that's where the FORWARDER fit in. – fortytwo Jul 08 '14 at 22:53
  • Seems like a reasonable choice. If you haven't read [the guide](http://zguide.zeromq.org/page:all) yet, I highly recommend you do so... if you've got a couple weeks to work through it at a natural pace and absorb it along the way, it'll answer most of your questions. In particular, you mention "create ZeroMQ contexts"... typically you will ever only need one per process, and all sockets will belong to that context. You can connect multiple sockets to each bound socket, so you may not even need to spin up multiple sockets on demand. Things like this could vastly simplify your architecture. – Jason Jul 09 '14 at 13:27
  • @Raggs > Text pointed by Jason is very informative. Peter Hintjens has published a book ( Ref. the PDF-link below ), which everybody who feels a need to know & _understand_ where the ZeroMQ immense power is & how to harness it for ones Architecture. No sample or library man-page will ever raise your vision of the power "behind" ... Worth see & taste **Fig.60 + Fig.62 to smell the smoking gun** inside ZeroMQ. Your views on architecture will forget atomic details & will benefit from the most powerfull concepts of Scaleable Formal Communications Patterns **_YOU_ may engineer-in** – user3666197 Jul 09 '14 at 14:22

2 Answers2

8

Answers

  1. Yes, ZeroMQ is capable of serving this need

  2. Yes. ZeroMQ is a right tool ( rather a powerful tool-box of low-latency components ) for this. While nanomsg has a straight primitive for bus, the core distributed logic can be integrated in ZeroMQ framework

  3. Yes & No. PUB-SUB as given above may serve for emulation of the "shout-cast"-to-bus and build on a SUB side-effect of using a subscription key(s). The WHOLE REST of the logic has to be re-thought and designed so as the whole scope of the fabrication meets your plans (ref. below). Also kindly bear in mind, that initial versions of ZeroMQ operated PUB/SUB primitive as "subscription filtering" of the incoming stream of messages being done on receiver side, so massive designs shall check against traffic-volumes / risk-of-flooding / process-inefficiency on the massive scale...

  4. Yes. ZeroMQ is rather a well-tuned foundation of primitive elements ( as far as the architecture is discussed, not the power & performance thereof ) to build more clever, more robust & almost-linearly-scaleable Formal Communication Pattern(s). Do not get stuck to PUB/SUB or PAIR primitives once sketching Architecture. Any design will remain poor if one forgets where the True Powers comes from.

A good place to start a next step forward towards a scaleable & fault-resilient Bus

Thus a best next step one may do is IMHO to get a bit more global view, which may sound complicated for the first few things one tries to code with ZeroMQ, but if you at least jump to the page 265 of the Code Connected, Volume 1, if it were not the case of reading step-by-step thereto.

The fastest-ever learning-curve would be to have first an un-exposed view on the Fig.60 Republishing Updates and Fig.62 HA Clone Server pair for a possible High-availability approach and then go back to the roots, elements and details.

halfer
  • 19,824
  • 17
  • 99
  • 186
user3666197
  • 1
  • 6
  • 50
  • 92
  • 2
    Thank you for the information and pointing me in the right direction. I have started reading the book. Up until now I referred the book here and there but this is the first time I am taking it step by step. I will post an update once I sort things out. – fortytwo Jul 09 '14 at 21:14
  • @Raggs > Glad to hear that, Raggs. The stories inside the book would never be "assembled" from the typical man-pages, the less from code-snippets flying in URL-HyperSpace. – user3666197 Jul 10 '14 at 23:58
  • All of the links included in this answer return 404 (Not Found) as of writing this comment. – Rikki Jan 11 '21 at 04:06
4

Here is what I ended up designing, if anyone is interested. Thanks everyone for the tips and pointers.

  1. I have a message bus implemented using ZeroMQ (and CZMQ) running as a separate process.
  2. The pattern is PUBLISHER-SUBSCRIBER with a LISTENER. They are connected using a PROXY.
  3. In addition, there is a ROUTER invoked using a newly forked thread.
  4. These three endpoints run on TCP and are bound to predefined ports which the clients know of.
  5. PUBLISHER accepts all messages from clients.
  6. SUBSCRIBER sends messages with a unique tag to the client who have subscribed to that tag.
  7. LISTENER listens to all messages passing through. currently, this is for logging testing and purposes.
  8. ROUTER provides a separate comms channel to clients. Messages such as control commands are directed here so that they will not get passed downstream.
  9. Clients connect to,
    1. PUBLISHER to send messages.
    2. SUBSCRIBER to receive messages. Subscription is using unique tags.
    3. ROUTER to send commands (check tag uniqueness etc.)

I am still doing implementation so there may be unseen problems, but right now it works fine. Also, there may be a more elegant way but I didn't want to throw away the PUB-SUB thing I had built.

fortytwo
  • 491
  • 1
  • 5
  • 16