3

I am using the C# wrapper for ZeroMQ but this seems more like an underlying issue with ZeroMQ.

Is there any way to push a message without blocking and without queueing? If the server is not up I would like the messages to be permanently disposed without blocking.

Here are the settings I've tried so far:

1)

Send (Blocking send)

High water mark = 0

This (stragely) does not block, but it seems to queue in memory until the socket is connected (memory keeps rising for the process).

2)

Send (Non-Blocking send)

High water mark = 1

This is a race condition. If I send two messages in rapid succession one message is sometimes thrown out for exceeding the high water mark.

3)

Poll the socket to figure out if it's going to block. This doesn't really help because I still have to put one (old) message in the queue before it starts blocking (if I set HWM = 1).

Non-blocking send with any high water mark is undesirable because as soon as the server comes back online it gets a bunch of old messages from clients.

Blocking send doesn't work because I don't want to block.

adotout
  • 1,170
  • 12
  • 17

1 Answers1

7

What you seem to be looking for is simply a PUB socket. This socket type never blocks on send, and discards any message it cannot send to a subscriber. See this page : http://api.zeromq.org/3-2:zmq-socket .

On a side note, you do not need to use this socket for "real" pub/sub, you can use it for nonblocking communication between two nodes by having only one PUB and one SUB socket by endpoint. Your server will not get "old" messages after a reconnect because the PUB sockets will have dropped the messages it could not send while the server was disconnected. Nevertheless i believe that while you cannot avoid some internal ZMQ "queuing", it should have little bearing on your use case.
JJ15k
  • 520
  • 6
  • 12
  • This certainly seems like an improvement in terms of message persistence. Unfortunately my system is such that I have multiple senders and one receiver, which appears to be incompatible with PUB/SUB without a lot of [hackery](http://stackoverflow.com/questions/6700149/python-zeromq-multiple-publishers-to-a-single-subscriber). – adotout Dec 12 '12 at 15:10
  • 2
    Lots of hackery? Not really, what you can do is Bind the subscriber(server), and connect all the publishers (workers) to the same endpoint. Make sure you only send messages after the connect is done (otherwise you may lose some) – JJ15k Dec 12 '12 at 18:41
  • Ah, it wasn't clear until I read a little closer that either side could bind/connect. How is this different from my #2 from above? That also did not block. The problem is that when I set the high water mark to something low (I don't want to build up old messages) it ends up throwing out messages even when the socket is connected. My current solution is to send a time stamp with every message, and throw out old messages (presumably that were stuck in the message queue). But I'd really rather not have to serialize/deserialize a timestamp for every message. – adotout Dec 12 '12 at 19:05
  • Why would you set a low HWM in this case? (and by low i assume something like your example, ie <10 messages) – JJ15k Dec 12 '12 at 19:21
  • 1
    If I understand the messages are stale if not immediately received, but the ZMQ architecture is pushing them as fast as it can, even if there is some "queuing" going on (which is actually good for you because it batches messages). Even if you go with raw sockets you may still have to check some timestamp to see if the message is too old (i'm assuming milli/microsecond latency here), because you have no real "control" over network latency. – JJ15k Dec 12 '12 at 19:29