Testing using inverted zeromq pub-sub in python

Question

I did use pyzmq 2.2.0.1 (python27 on Windows or Linux) in my code and when I running this it works (also it python threads):

def test_zmq_inverted_pub_sub():
    import zmq
    import time
    ctx = zmq.Context()
    sub = ctx.socket(zmq.SUB)
    pub = ctx.socket(zmq.PUB)
    sub.bind('tcp://127.0.0.1:5555')
    sub.setsockopt(zmq.SUBSCRIBE, b'')
    time.sleep(3)
    pub.connect('tcp://127.0.0.1:5555')
    pub.send(b'0')
    assert sub.poll(3)

When I'd upgrade my zmq to 13.1.0 (and now to 14.0.0) I see this test doesn't work.

I tried searching some changes about it but I didn't find. When I creating this queues on different processes it's work but I don't want to open new process for my test. is there any explanation why it's doesn't work and how can I do this test right?

Thanks.

score 1 · Accepted Answer · answered Nov 19 '13 at 01:35

1

This is mainly because subscriptions are filtered PUB-side, starting with zeromq 3.0. It takes a finite time for subscriptions to propagate, so the fact that you are trying to send immediately after you establish the connection means that you are probably sending before the PUB socket knows that it has any subscribers.

There is a secondary issue that is a known bug, specific to when SUB binds and PUB connects. The result is that the SUB socket does not tell the PUB about its subscriptions until the first time it polls / recvs after the connection has been established.

This version of the test will pass:

def test_zmq_inverted_pub_sub():
    import zmq
    import time
    ctx = zmq.Context()
    sub = ctx.socket(zmq.SUB)
    pub = ctx.socket(zmq.PUB)
    sub.bind('tcp://127.0.0.1:5555')
    sub.setsockopt(zmq.SUBSCRIBE, b'')
    pub.connect('tcp://127.0.0.1:5555')
    # the first sub.poll is a workaround to force subscription propagation
    for i in range(2):
        pub.send(b'hi')
        evt = sub.poll(1)
        if evt:
            break
    assert evt

answered Nov 19 '13 at 01:35

minrk

37,545
9
92
87

Thank you. it's nice to understand the problem. i wounder if there is a way to do this without sending message twice. there is a logic problem for making pub waiting for sub poll, so it's not really working for me. – netanelrevah Nov 21 '13 at 17:53
you don't have to send the message twice, you only have to call poll or recv twice. The advantage of sending twice here is that there is no time.sleep, which you would need if you only had the extra recv. – minrk Nov 21 '13 at 20:47
but if I have a subscriber that bound to some port and I have create in another process a publisher it can't know where the subscriber did the first poll and waiting for this before send it will send the first message so I thought sending twice can solve this but it's no practical solution. – netanelrevah Nov 22 '13 at 09:31
as long as the sub is polling, you should be okay. I don't know what your publishers actually do, but a short time.sleep prior to their first send should make subscriptions likely to have propagated, assuming the subscriber is waiting inside `poll()`. – minrk Nov 22 '13 at 19:11

Testing using inverted zeromq pub-sub in python

1 Answers1