I am working on a requirement whereby a process (say producer) needs to send out one-way messages to a variable number of processes (say consumers).
The publish-subscribe model seemed good for this because the consumers will subscribe to messages from the producer. I tried using ZeroMQ to achieve this.
However, I have a few problems with it:
The consumers have to continuously poll for messages. I would have the consumers to be notified when there is a new message.
There is a possibility of the producer queue being filled up. I would have liked the producer to remove messages from the queue based on some condition (say remove messages older than 5 seconds, or remove messages that have been read 5 times).
Since the consumers are polling and the messages are not removed from the queue, the consumers see duplicate messages till a new message comes in. I want the consumer to be notified only once per new message.
I understand I may be using a wrong model (publish-subscribe may not be suitable). I have thought about using request-reply, but that doesn't work since the producer does not want to keep track of the number of consumers.
Can anyone suggest a good alternative?