How to implement proxy/broker for (X)PUB/(X)SUB messaging in ZMQ?

Question

So I was reading this article on how to create proxy/broker for (X)PUB/(X)SUB messaging in ZMQ. There is this nice picture of what shall architecture look like :

data flow: user code -> PUB -> XSUB -> user code-> XPUB -> SUB -> user code; subscription flow: user code <- PUB <- XSUB <- user code <- XPUB <- SUB <- user code;

But when I look at XSUB socket description I do not get how to forward all subscriptions via it due to the fact that its Outgoing routing strategy is N/A

So how one shall implement (un)subscription forwarding in ZeroMQ, what is minimal user code for such forwarding application (one that can be inserted between simple Publisher and Subscriber samples)?

minrk · Accepted Answer · 2013-08-15T05:39:59.830

18

XPUB does receive messages - the only messages it receives are subscriptions from connected subscribers, and these messages should be forwarded upstream as-is via XSUB.

The very simplest way to relay messages is with zmq_proxy:

xpub = ctx.socket(zmq.XPUB)
xpub.bind(xpub_url)
xsub = ctx.socket(zmq.XSUB)
xsub.bind(xsub_url)
pub = ctx.socket(zmq.PUB)
pub.bind(pub_url)
zmq.proxy(xpub, xsub, pub)

which will relay messages to/from xpub and xsub. Optionally, you can add a PUB socket to monitor the traffic that passes through in either direction.

If you want user code in the middle to implement extra routing logic, you would do something like this, which re-implements the inner loop of zmq_proxy:

def broker(ctx):
    xpub = ctx.socket(zmq.XPUB)
    xpub.bind(xpub_url)
    xsub = ctx.socket(zmq.XSUB)
    xsub.bind(xsub_url)

    poller = zmq.Poller()
    poller.register(xpub, zmq.POLLIN)
    poller.register(xsub, zmq.POLLIN)
    while True:
        events = dict(poller.poll(1000))
        if xpub in events:
            message = xpub.recv_multipart()
            print "[BROKER] subscription message: %r" % message[0]
            xsub.send_multipart(message)
        if xsub in events:
            message = xsub.recv_multipart()
            # print "publishing message: %r" % message
            xpub.send_multipart(message)

        # insert user code here

full working (Python) example

edited Aug 15 '13 at 05:39

answered Jan 29 '13 at 21:21

minrk

37,545
9
92
87

You can replace everything from `poller = …` down with `zmq.proxy(xsub, xpub)` (`zmq_proxy()` in C). The only thing is, to see messages in transit, you have pass in a capture socket as a third parameter, which is a bit more involved. – Marcelo Cantos Aug 14 '13 at 00:34
That misses the point, which is the addition of user code inside the device. If you don't need code in between the XPUB and XSUB, a proxy works just fine (or in Python you can use monitored_queue, zmq_proxy's predecessor). – minrk Aug 15 '13 at 04:38
From the phrase, “what is minimal user code for such forwarding application”, it seems pretty clear to me that the OP doesn't have some block of code they want to inject between the XSUB and XPUB. They just want a minimal forwarding solution. Even so, it's perfectly OK to offer the more flexible solution, but the OP should be made aware that a much simpler solution exists if they don't need to play MITM. – Marcelo Cantos Aug 15 '13 at 05:26
Makes sense, I will update the answer with clarification about when a manual broker is appropriate. – minrk Aug 15 '13 at 05:35
1

Answer of a C++ question in Python - great! One cannot think high enough of the job done my ZMQ folks in regards to language porting. – Patrick B. Jan 30 '17 at 08:07

How to implement proxy/broker for (X)PUB/(X)SUB messaging in ZMQ?

1 Answers1