3

I'm new to RabbitMQ / EasyNetQ and am trying to better understand a behaviour I am observing. We've seen that when our server running RabbitMQ is busy all EasyNetQ connections are dropped.

This is the exception simultaneously generated on all clients:

System.Exception: Failed to connect to Broker: 'XXXXXX.domain.com', Port: 5672 VHost: 'XXXX'. ExceptionMessage: 'None of the specified endpoints were reachable'

EasyNetQ automatically reconnects when the server is no longer busy, but I wonder if it is typical for RabbitMQ/EasyNetQ to drop connections when the machine is busy? (Or if I should be investigating performance issues with my server.)

(PS: By busy, I simply mean updating a large project from source control, relaunching a large ASP.NET application after redeploying it or running a CPU-intensive calculation on large amounts of data. ).

Jason Kealey
  • 7,988
  • 11
  • 42
  • 55

1 Answers1

3

There are limits to the number of connections a RabbitMQ broker will accept. Is it possible that you are rapidly opening a connection, doing some work, then closing it, much as you would with a database connection? If so, that's not how you should interact with the broker. See the EasyNetQ documentation on connections:

https://github.com/mikehadlow/EasyNetQ/wiki/Connecting-to-RabbitMQ

Mike Hadlow
  • 9,427
  • 4
  • 45
  • 37
  • Thanks for the response, Mike. We're talking about two dozen connections in total (one per peer), which are created when my app boots up and disposed when it shuts down (always connected). This shouldn't be the issue. – Jason Kealey Jun 10 '14 at 12:08
  • It sounds like you're doing the right thing. I don't have any other suggestions as far as EasyNetQ use goes. I'd recommend describing your problem on RabbitMQ mailing list. Give as much information as you can. – Mike Hadlow Jun 10 '14 at 13:18
  • Thanks. However, because you've played around with RabbitMQ a lot before, have you personally seen connections drop when the server has a "high" load? Seems abnormal to me. I may be investigating a symptom when the root cause may be server performance. – Jason Kealey Jun 10 '14 at 13:22
  • Nope, I've never seen connections just drop for no reason, and that's on a production system processing millions of messages per day. Do you have the EasyNetQ logs showing a disconnection? – Mike Hadlow Jun 10 '14 at 14:34
  • Unfortunately, the exception posted in the original comment is the only logging I have. Our IEasyNetQLogger dispatched that exception to us via Elmah. – Jason Kealey Jun 11 '14 at 00:08
  • You're going to find life very difficult if you don't log the EasyNetQ output somewhere. It's the first place to look for problems like this. – Mike Hadlow Jun 11 '14 at 10:19
  • Turns out I was wrong; we do log more than exceptions in another location. Still, not much more info in here. Here's the inverse sequence of events (last to first): Trying to connect Disconnected from RabbitMQ Broker Persistent channel disconnected. Trying to connect Disconnected from RabbitMQ Broker Persistent channel disconnected. Trying to connect Disconnected from RabbitMQ Broker Persistent channel disconnected. – Jason Kealey Jun 11 '14 at 14:11
  • Hi, did you ever get to the bottom of this? – Matt Mar 18 '21 at 11:51