0MQ: Can you drop a message after a timeout in a REQ/REP pattern?

Question

In my 0MQ applications I usually do this to deal with a timeout:

import zmq

ctx = zmq.Context()

s = ctx.socket(zmq.DEALER)
s.connect("tcp://localhost:5555")

# send PING request
v = <some unique value>
s.send_multipart(["PING", v])
if s.poll(timeout * 1000) & zmq.POLLIN:
    msg = s.recv_multipart()
    ...

However if the server is not running and minutes later goes online, then 0MQ will automatically reconnect and send the message. However if I put the send PING command in a loop (once a second) and the server is down, then once the server goes back online I'll get on the next recv_multipart()-call the old messages that remained in the internal 0MQ queue while the server was offline. Because I don't care about old messages, I though I could do this:


s = ctx.socket(zmq.DEALER)
s.connect("tcp://localhost:5555")

while True:
    # send PING request
    v = <some unique value>
    s.send_multipart(["PING", v])
    if s.poll(timeout * 1000) & zmq.POLLIN:
        msg = s.recv_multipart()
        ...
    else:
        s.close()
        s = ctx.socket(zmq.DEALER)
        s.connect("tcp://localhost:5555")

    time.sleep(1)

But this is a bad idea, after a while ctx.socket raises ZMQError: Too many open files. Setting the ZMQ_LINGER socket option to 0 seems to help here, but now I don't like this strategy, seems wrong to me.

So ideally I'd like to drop the previously sent message if a read timeout happens. Is this a) possible and b) a good idea at all? I'm not sure that this would be correct though, it may be that 0MQ is able to physically send the message but the server crashes before it can send back anything, so dropping would be impossible because there wouldn't be anything to drop, would it?

So my question is: what should I do in this situation? Am I possibly looking at this problem from the wrong angle?

score 2 · Accepted Answer · answered Feb 20 '20 at 18:30

2

Q : "Can you drop a message after a timeout in a REQ/REP pattern?"

No.

Q : Is this a) possible and b) a good idea at all?

a) Yes.
b) Yes.

Q : what should I do in this situation?

Best avoid the REQ/REP certainty of falling into a mutual deadlock ( you just never know when it happens - many posts with details here ) + setup the connections layer so as to provide self-defensive means for delivering only the last message over a healthy connection :

...
s = ctx.socket( zmq.DEALER )

s.setsockopt( zmq.LINGER,    0 ) # ALWAYS, no excuse, you never know the peers' versions
s.setsockopt( zmq.IMMEDIATE, 1 ) # prevent sending over incomplete yet connections
s.setsockopt( zmq.CONFLATE,  1 ) # Does not support multi-part, so .pack() payload

s.connect( "tcp://localhost:5555" )
...

answered Feb 20 '20 at 18:30

user3666197

1
6
50
92

Thanks for the answer, but I'm not sure I understand it completely. If I set the `zmq.IMMEDIATE` socket option, then does it mean that `s.send` would block until the server is really online? – Pablo Feb 21 '20 at 10:32
`send()`-method blocks if & "only" if you want it to block, still but under some particular local `Context()`-instance conditions (ref. API for details on this). Professional systems should never block ( **Who would ever wish to loose one's control over the owned domain-of-control?** Would you consider it wise to become and remain without any chance to change such **state of your *accepted* power-less-ness** ? *"Powerlessness of Empowered" is a nice philosophy work, but let it be in control-systems?* ) avoid any blocking form of the `{ .send() | .recv() }`-methods, using the flag `zmq.NOBLOCK` – user3666197 Feb 21 '20 at 13:39

0MQ: Can you drop a message after a timeout in a REQ/REP pattern?

1 Answers1