1

We have a bunch of requests that we plan to publish to the queue.

There will be several different subscriber types, each in their own round robin pool.

For example Request1 is pushed onto the queue

LoggingSubscriber1 and LoggingSubscriber2 both subscribe with the "LoggingSubscriber" subscriptionId so that only one of them gets the request.

There will be other groups like DoProcessSubscriber1, DoProcessSubscriber2, and DoProcessSubscriber3

And another DoOtherProcessSubscriber1, DoOtherProcessSubscriber2

We need some way to know that all three subscribers (Logging, DoProcess, and DoOtherProcess) have completed, so that we can perform some action...like sending a message to the client that all the entire request has completed.

How would we aggregate responses like this? We were thinking of having each subscriber put a response object on the queue, but we still aren't sure how to know that they are all done.

CaffGeek
  • 21,856
  • 17
  • 100
  • 184
  • I suggest you to use some kind of counter in RequestObject, so each subscriber will increment it as soon as finished work, then you can compare this counter with expected value (for you - 3) and decide. Don't forget that this counter will be a shareable resource in multithreaded environment, so need some kind of synchronization – Iłya Bursov Oct 11 '13 at 21:14
  • I don't want a hardcoded value, if we add another subscriber type we shouldn't ideally need to modify a config an expected value. I believe items stay on the Queue until all subscribers have ACKed correct? So is there not a way to have something happen at that time? – CaffGeek Oct 11 '13 at 21:25
  • As you know how many subscribers in your list - compare not with hardcoded value, but with this length – Iłya Bursov Oct 11 '13 at 21:27
  • I don't know how many subscribers are in my list. That's the problem. The subscribers are different applications. – CaffGeek Oct 11 '13 at 21:31
  • According to [manual](http://hg.rabbitmq.com/rabbitmq-management/raw-file/rabbitmq_v3_1_5/priv/www/api/index.html) you can find out who connected – Iłya Bursov Oct 11 '13 at 21:42

1 Answers1

1

Ideally you'd use the Request/Response pattern built into EasyNetQ, but that's designed for a single (potentially farmed) consumer. It doesn't allow you to bind to multiple queues. In your case you should probably have your client set up a subscription for replies and have all three services publish a message when they are complete. The client can then wait until it has a response from all three before updating.

However, I'd encourage you to possibly re-think your design. By making the client responsible for acknowledging the completion of the subscribers, you're building a very tightly coupled system. Messaging System design works far better if you adopt the notion of eventual consistency. Allow your client to fire-and-forget and have some audit process ensure that all the expected processing did eventually occur.

Mike Hadlow
  • 9,427
  • 4
  • 45
  • 37