1

I have a ZMQ_PUB socket sending messages out at ~50Hz. One destination needs to react to each message, so it has a standard ZMQ_SUB socket with a while(true) loop checking for new messages. A second destination should only react once a second to the "most recent" message. That is, my second destination needs to subsample.

For the second destination, I believe I'd want to have a time-based loop that is called at my desired rate (1Hz) and recv() the latest message, dropping the rest. I believe this is done via a ZMQ_HWM on the subscriber. Is there another option that needs to be set somewhere?

Do I need to worry about the different subscribers having different HWMs? Will the publisher become angry? It's a shame ZMQ_RATE only applies to multicast sockets.

Is there a best way to accomplish what I'm attempting?

zmq v3.2.4

Dustin
  • 73
  • 1
  • 6

1 Answers1

0

The HighWaterMark will not be a fantastic solution for your problem. Setting it on the subscriber to, let's say, 10 and reading 1 message per second, will just give you the old messages first, slowly, and throw away all the new, because it's limit are reached.

You could either use a topic on you publisher that makes you able to filter out every 50th message like making the topic messageCount % 50 and subscribe to 0.

Otherwise maybe you shouldn't use zmq's pub/sub, but instead do you own look alike with router/dealer that allows you to subscribe to sampled messages.

Lastly you could also just send them all. 50 m/s is hardly anything in zmq (if they aren't heavy on data, like megs) and then only use every 50th message.

Emil Ingerslev
  • 4,645
  • 2
  • 24
  • 18
  • I'm new to SO, so maybe there's a better way to respond, but I found a solution that works for my needs (though perhaps not ideal). I added an intermediate object that subscribed to every message, storing the most recent to a member variable, and periodically re-published whatever happened to be stored last. I have both cpp and python implementations. Then, wherever I want to receive sampled data, I instantiate this object in front of it and spin it. The destination doesn't notice any difference in content, only rate. – Dustin May 11 '15 at 15:56
  • Interesting solution. Do you have multiple `intermediate objects` or consider having multiple of them? – Emil Ingerslev May 11 '15 at 20:55
  • There's nothing preventing me from having multiple "rate limiting" objects. I could deploy them in front of every end point, or as one-to-many republishers for any end point that requires a specific rate (one running at 100Hz, another at 50Hz). I'm happy to post the code. Do I do that here in a comment, or do I edit my question? – Dustin May 12 '15 at 13:24
  • I think I would be easier to add it as an edit to the answer. Otherwise reference a gist. – Emil Ingerslev May 17 '15 at 07:36