AMQ + Oracle AQ: lost message on failed connection

Question

I've configured AMQ to receive and send messages from Oracle AQ, as is is described here http://activemq.apache.org/jms-bridge-with-oracle-aq.html

AMQ creates and holds connection to receive messages from Oracle AQ - this is correct. However, the new connection has been created and destroyed each time when AMQ sends a message to Oracle AQ. Sometimes, due to some network issues or Oracle overloaded, I get an exception:

org.apache.camel.RuntimeCamelException: org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is oracle.jms.AQjmsException: IO Error: Socket read timed out
        at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1344)
        at org.apache.camel.component.jms.EndpointMessageListener$EndpointMessageListenerAsyncCallback.done(EndpointMessageListener.java:186)
        at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:107)
        at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:562)
        at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:500)
        at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:468)
        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:325)
        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1103)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1095)
        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:992)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is oracle.jms.AQjmsException: IO Error: Socket read timed out
        at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
        at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
        at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.send(JmsConfiguration.java:226)
        at org.apache.camel.component.jms.JmsProducer.doSend(JmsProducer.java:415)
        at org.apache.camel.component.jms.JmsProducer.processInOnly(JmsProducer.java:369)
        at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:145)
        at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:110)
        at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)
        at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
        at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
        at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
        at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:105)
        at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:87)
        at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:103)
        ... 11 more
Caused by: oracle.jms.AQjmsException: IO Error: Socket read timed out
        at oracle.jms.AQjmsDBConnMgr.checkForSecurityException(AQjmsDBConnMgr.java:939)
        at oracle.jms.AQjmsDBConnMgr.getConnection(AQjmsDBConnMgr.java:617)
        at oracle.jms.AQjmsDBConnMgr.<init>(AQjmsDBConnMgr.java:251)
        at oracle.jms.AQjmsConnection.<init>(AQjmsConnection.java:185)
        at oracle.jms.AQjmsConnectionFactory.createConnection(AQjmsConnectionFactory.java:583)
        at org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.doCreateConnection(UserCredentialsConnectionFactoryAdapter.java:175)
        at org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.createConnection(UserCredentialsConnectionFactoryAdapter.java:150)
        at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:456)
        ... 23 more
Caused by: java.sql.SQLRecoverableException: IO Error: Socket read timed out
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:458)
        at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
        at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
        at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:187)
        at oracle.jms.AQjmsDBConnMgr.getConnection(AQjmsDBConnMgr.java:579)
        ... 30 more
Caused by: oracle.net.ns.NetException: Socket read timed out
        at oracle.net.ns.Packet.receive(Packet.java:339)
        at oracle.net.ns.NSProtocol.connect(NSProtocol.java:296)
        at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1102)
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:320)
        ... 37 more

And after that the message is lost.

Due to new connection operation is rather heavy, I've tried to configure org.springframework.jms.core.JmsTemplate to use pooled connections as it is described here http://activemq.apache.org/spring-support.html But this hasn't helped. New connections are still being created.

Please, advice the method which would help me not to loose messages from AMQ to Oracle AQ.

score 0 · Answer 1 · answered May 09 '18 at 18:59

It might be too late for your question. However, I want to share some thought about this problem.

The example only shows how to read from Oracle AQ to AMQ. I didn't see the logic about sending to Oracle AQ. By looking at your exception message, it seems you created an Oracle AQ client to send the message. In this case, it just opens a connection and dumps the message. As you can see, if it failed, this connection itself won't preserve the message.

So a couple of things you can do:

Continue to use Oracle AQ client and handle the failure/retry in your code. This means if sending to Oracle AQ failed, let the code retry later. Make sure the code close the connection every time.
You already have a local AMQ broker. I am not familiar with Oracle AQ. But it might be a network broker model. Refer to here and set duplex=true. Send the message to your local AMQ broker. Then if it failed to connect to the remote broker, the message will stay locally and be sent out when connection is reestablished.

AMQ + Oracle AQ: lost message on failed connection

1 Answers1