Mule ESB + RabbitMQ reliability and up-time: how to re-create a deleted exchange automatically at runtime

Question

I've got a Large mule application that receives on multiple queues and publishes to at least one exchange. I'm manually QA'ing the application by deleting the queues and exchanges to see if mule will retry to connect (and also shut down rabbit altogether)

This question is for when an exchange is deleted. The messages go into a locked anonymous queue named something like: amq.gen-gFs6-7sP2nw1ntgobO6cBg

I'm looking for a way to reconnect the exchange and still pass through the messages. Is this even possible?

I've set options like exchangeDurable="true" queueDurable="true"

Is there any other things (or rabbit configs) that I need to do or would be beneficial to me?

Next Test: Shutdown RabbitMQ in the middle of processing.

Code:

<mule xmlns:jdbc-ee="http://www.mulesoft.org/schema/mule/ee/jdbc"
xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns="http://www.mulesoft.org/schema/mule/core" 
xmlns:amqp="http://www.mulesoft.org/schema/mule/amqp" 
xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" 
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.4.2" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xmlns:mulexml="http://www.mulesoft.org/schema/mule/xml" 
xmlns:file="http://www.mulesoft.org/schema/mule/file" 
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/jdbc http://www.mulesoft.org/schema/mule/ee/jdbc/current/mule-jdbc-ee.xsd
http://www.mulesoft.org/schema/mule/xml http://www.mulesoft.org/schema/mule/xml/current/mule-xml.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/amqp http://www.mulesoft.org/schema/mule/amqp/current/mule-amqp.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd"
xmlns:context="http://www.springframework.org/schema/context">

<amqp:connector name="amqpAutoAckLocalhostConnector"
      host="${config.status_reporting.host}"
      port="${config.status_reporting.port}"
      virtualHost="${config.status_reporting.virtual_host}"
      username="${config.status_reporting.username}"
      password="${config.status_reporting.password}"
      requestedHeartbeat="${config.status_reporting.requestedHeartbeat}"
      doc:name="AMQP Connector for Status Messages"/>



<flow name="send_status" doc:name="send_status">        
    <vm:inbound-endpoint path="send_status" exchange-pattern="one-way" responseTimeout="10000" doc:name="VM" />
    <logger message="starting send status" level="DEBUG" doc:name="Logger"/>

    <!-- some code here has been removed for stackoverflow question --> 

    <flow-ref name="cwm_send" doc:name="flow ref"/>
</flow>



<flow name="cwm_send" doc:name="cwm_send">
    <amqp:outbound-endpoint exchangeName="${config.status_reporting.exchange_name}"
                      exchangeType="topic"                        
                      exchangeDurable="${config.status_reporting.exchange_is_durable}"
                      routingKey="${config.status_reporting.routing_key}"
                      connector-ref="amqpAutoAckLocalhostConnector" doc:name="AMQP Out" queueDurable="true" responseTimeout="10000"/>          
</flow>



<flow name="send_ingest_status" doc:name="send_ingest_status">
    <vm:inbound-endpoint exchange-pattern="request-response" responseTimeout="10000" doc:name="VM" path="send_ingest_status"/>
    <vm:outbound-endpoint path="send_status" exchange-pattern="one-way" doc:name="Send Status">
        <set-payload value="#[[
            'status_code': 'foo',
            'status_descr': 'test description.',
            'status_final': '0',    
            'version': '1.0']]"/>
    </vm:outbound-endpoint>

</flow>
</mule>

Thank you.

Answer 1

This problem also seems to occur when an exchange is declared only in an outbound endpoint. There is an open bug concerning this in the Mulesoft JIRA, and you can vote for it to help them prioritize it.

I took a look at the source code, and the problem seems to be that there is simply no code to declare exchanges when an outbound endpoint is started. In your case, you'd probably want the code to run at the time the message is sent, or maybe at the time the exchange is deleted. This timing wouldn't be covered by the aforementioned bug, but you might open a new issue describing the use case and the desired functionality. And a pull request would probably be even better! ;)