-2

I have a scenario where publishers need to send messages to a known exchange (they won't know the downstream queues directly). Most of the messages should be delivered to every bound queue (like a fanout), but some need to be delivered only to a specific bound queue. I'd like to avoid multiple exchanges as well as multiple queue bonds.

My current solution uses header routing, where a message has one of two headers, identify it as "global" or "specific to a particular sub", where the downstream queues are bound on a match-any to both of those headers. I think this will work, but I feel like there should be a simpler solution.

I tried to find an exchange plugin that would "fanout all messages except specific ones", but I couldn't find such a plugin. Outside of that, any ideas on how to implement such a routing strategy?

eric
  • 511
  • 1
  • 4
  • 15
  • Not sure why you're getting downvoted, but I would think a topic exchange would be the best option for this. – mshindal Oct 10 '17 at 15:19
  • @mshindal - Not sure why either; it seems like a reasonable question for a legitimate use-case (and I've done tons of research). With regard to your suggestion, I'm not sure how to make this work with a topic exchange, unless I bind two queues for each client (one for "all messages" and one for "specific messages"). Is two queues what you had in mind? – eric Oct 11 '17 at 02:31

2 Answers2

0

For what it's worth, my original solution using a "headers" exchange with queues bound using "match any" is the only one I could find, short of writing a new exchange plugin. It does work and so far seems reasonably fast (at least not measurably slower than a typical "topic" exchange--which I could find no way to apply in this scenario).

From my research on this topic, the ideal solution would be a "topic" exchange with the ability to use a RegEx or at least some form of "or-logic". I did find some information that implied that a RegEx was considered but decided against (in favor of the "dotted notation" topic format) because the latter was faster, specifically because the use a a RegEx would require that every binding be evaluated on each new message (i.e. there was no way to construct a "search shortcut").

For now, my "match-any headers exchange" solution will serve my purposes, but in the future, a "topic" exchange that allowed "or-logic" might be worth exploring. It would allow multiple topic patterns to be achieved with single-binding, without the overhead of a RegEx. But I have no experience with Erlang, nor the time to learn enough of it to write the necessary plugin. Please contact me if anyone is interested in collaborating on this.

eric
  • 511
  • 1
  • 4
  • 15
0

What you describe really sounds like exchange-to-exchange binding though, as rich routing is actually one of RMQ's strengths.

You could create the entry exchange that would point to either the fanout (for generic case) or topic/direct one (for the special one), and leave all routing to RMQ. The entry-exchange could be a header exchange or a direct one, depending on what you want to put in headers:

entry-exchange -----> fanout-exchange ---*---> multiple-fanout queues
               \
                \---> the special exchange --> queue-for-special-usecase
Adam Kotwasinski
  • 4,377
  • 3
  • 17
  • 40
  • Yes, I had taken a hard look at using bound exchanges, but all solutions I found (including what I believe you're suggesting) would require the clients to be bound to two different queues (one for fanout and one for direct). This is something I was specifically trying to avoid, as there will be a large number of clients, so I didn't want to multiply that by 2X. – eric Oct 13 '17 at 14:20