2

The failover transport in ActiveMQ works nicely if one of broker instances goes down - producer automagically switches to the next broker instance what is good.

I was expecting the same behavior if broker reaches it's memory limit (configured through systemUsage property in activemq.xml). However in this case it just starts to to throw 'Memory is full' warnings in the log, but jmsTemplate.convertAndSend method in producer's application just hangs.

I tried to set sendFailIfNoSpace="true" option for systemUsage property. In this case producer application just started to throw org.springframework.jms.ResourceAllocationException: Usage Manager Store is Full exception on convertAndSend and did not switch to good broker instance anyway.

Could someone advice how to fail-over on exception and / or 'memory is full' case?

adrift
  • 637
  • 2
  • 10
  • 34

1 Answers1

3

When a broker runs out of memory the producer is blocked until some memory is cleared. This is under the assumption that messages are consumed on a regular basis.

What really should worry you is why your queues are running out of memory.

Are you producing messages faster that you are consuming them? If so, failover won't really help you, as the next broker will run out of memory quickly. ActiveMQ supports producer flow control to make sure the producers are not overwhelming the broker. (see here)

What is happening to the consumed messages? Are they stored forever? If so, consider using timeToLive to make sure you clear space for incoming messages.

If, for some reason, you need to fill queue after queue, you can just catch the exception in the producer and manually failover.

Also see this similar question: ActiveMQ: Reject connections from producers when persistent store fills

Community
  • 1
  • 1
summerbulb
  • 5,709
  • 8
  • 37
  • 83
  • I just want to protect the system from potential message loose if something goes wrong on message consumption side (crash of consumer application, etc.), assuming that memory limit of backup brokers will be high enough. So, you are saying that the only option for this case is to catch "org.springframework.jms.ResourceAllocationException: Usage Manager Store is Full" and then programmatically switch to different broker? – adrift Oct 29 '13 at 18:09
  • @ViktorsOginskis I'm saying it's relatively easy and straightforward. Especially for the edge case you are talking about. – summerbulb Oct 29 '13 at 18:12