3

I've read many posts and blogs at this point and I'm still not sure about how to cluster correctly my 2 RabbitMQ nodes.

I've read the RabbitMQ clustering guide: http://www.rabbitmq.com/clustering.html

I found out about a mysterious ClusterId in the API guide, with no explanation on how to get that Id in the first place: http://www.rabbitmq.com/releases/rabbitmq-dotnet-client/v2.4.1/rabbitmq-dotnet-client-2.4.1-api-guide.pdf

Learned in that StackOverflow post that basically I'd need my clients to be aware of each node in the cluster and code for failover scenario: rabbitmq HA cluster

Now... The behavior I'd like to have is something a little more transparent if possible. Where I would potentially use that "ClusterId" on the client to make the consumer cluster aware and then hopefully the library knows to randomly connect to either node to grab messages.

Granted I know a message can only be on one server at a time, so I'm hoping for some round robin magic from the DotNet client library that would also handle fail over situations.

What I was hoping also from a publisher perspective is that the exchange would round robin distribute messages to the various nodes in the cluster. The exchange would also be cluster aware and handle fail over situations gracefully.

Now based on my readings it doesn't quite work like that... unless I missed something. If my knowledge is up to date and I have to code all that cluster aware business, then... why does RabbitMQ have a cluster feature in the first place? How is it used?

Is there a way to have that kind of behavior out of RabbitMQ without coding that much?

Thanks

Community
  • 1
  • 1
Lancelot
  • 2,417
  • 12
  • 39
  • 46
  • 1
    I was confused about RabbitMq clustering. I now have a better understanding of Erlang, RabbitMq and as to why the queues aren't replicated in the cluster and also how that does not impact scalability, which was my main goal here. All the nodes know where each queue lives, and forward messages. Therefore there is no need to have the same queue replicated unless the goal is high availability, which isn't the case for me. Thanks for your help Troydm. – Lancelot Jun 24 '11 at 15:57

1 Answers1

5

I'm using HAProxy for failover and load balancing between cluster nodes for rabbitmq, also on .net client side on exception when node fails you need to manually reconnect the client.

Here is the HAProxy configuration for two node cluster, nodes are running on 15672 and 25672 ports. Clients connect on 5672 port.

global
 daemon
 log 127.0.0.1 alert
 log 127.0.0.1 alert debug

defaults
 log global
 mode http
 option  dontlognull
 option  redispatch
 retries    3
 contimeout 5000
 clitimeout 50000
 srvtimeout 50000

listen rabbitmq 0.0.0.0:5672
  mode tcp
  balance roundrobin
  option  tcpka

server rabbit01 127.0.0.1:25672 check inter 5000 downinter 500

server rabbit02 127.0.0.1:15672 check inter 5000 backup

cudos to this blog post http://www.joshdevins.net/2010/04/16/rabbitmq-ha-testing-with-haproxy/

Troydm
  • 2,642
  • 3
  • 24
  • 35
  • I have followed the link you have shared. but the request is not distributed over two servers. messages publish only in one server. – Anand Soni May 11 '12 at 07:38
  • I don't think that messages published will be distributed, it's just for failover e.g. the situation when one main broker fails, the client will be reconnected to the second broker – Troydm May 11 '12 at 18:37
  • I have posted the question over stackoverflow. check this http://stackoverflow.com/questions/10547523/configure-haproxy-for-rabbitmq – Anand Soni May 12 '12 at 04:02