0

We have created a queue on the servicebus and the service crash when "Creating CBS link to $cbs". The service use a queue and a topic on the servicebus, however the problem only appear since we added the queue. In some case we don't have that problem but the service later hang and don't responds, its seem to be because the connection timeout.

Here the exception and trace at the startup

2019-01-11 09:51:53.916  INFO 5756 --- [ca-7e46fe9c28c4] c.m.a.s.primitives.MessagingFactory      : Creating CBS link to $cbs
2019-01-11 09:52:23.923 ERROR 5756 --- [pool-1-thread-5] c.m.a.s.primitives.RequestResponseLink   : RequestResponseLink open timed out.

com.microsoft.azure.servicebus.primitives.TimeoutException: Open operation on RequestResponseLink(MessagingFactory717191-cbs) on Entity($cbs) timed out at 2019-01-11T09:52:23.921251500-05:00[America/New_York].
    at com.microsoft.azure.servicebus.primitives.RequestResponseLink$1.run(RequestResponseLink.java:78)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

2019-01-11 09:52:53.815 ERROR 5756 --- [ca-7e46fe9c28c4] c.m.a.s.primitives.MessagingFactory      : Connection error. 'Error{condition=amqp:connection:forced, description='The connection was inactive for more than the allowed 60000 milliseconds and is closed by container 'LinkTracker'. TrackingId:9bbe750504544351b172806983132bef_G12, SystemTracker:gateway7, Timestamp:2019-01-11T14:53:52', info=null}'

2019-01-14 15:46:45.604  WARN 18908 --- [ef-8af7499a0297] c.m.a.s.primitives.RequestResponseLink   : Internal send link of requestresponselink to '$cbs' closed with error.

com.microsoft.azure.servicebus.primitives.ServiceBusException: Error{condition=amqp:connection:forced, description='The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:9bbe750504544351b172806983132bef_G12, SystemTracker:gateway7, Timestamp:2019-01-14T20:50:43', info=null}
    at com.microsoft.azure.servicebus.primitives.ExceptionUtil.toException(ExceptionUtil.java:113)
    at com.microsoft.azure.servicebus.primitives.RequestResponseLink$InternalSender.onClose(RequestResponseLink.java:837)
    at com.microsoft.azure.servicebus.amqp.BaseLinkHandler.processOnClose(BaseLinkHandler.java:68)
    at com.microsoft.azure.servicebus.amqp.BaseLinkHandler.onLinkRemoteClose(BaseLinkHandler.java:42)
    at org.apache.qpid.proton.engine.BaseHandler.handle(BaseHandler.java:176)
    at org.apache.qpid.proton.engine.impl.EventImpl.dispatch(EventImpl.java:108)
    at org.apache.qpid.proton.reactor.impl.ReactorImpl.dispatch(ReactorImpl.java:309)
    at org.apache.qpid.proton.reactor.impl.ReactorImpl.process(ReactorImpl.java:276)
    at com.microsoft.azure.servicebus.primitives.MessagingFactory$RunReactor.run(MessagingFactory.java:481)
    at java.base/java.lang.Thread.run(Unknown Source)
Chris
  • 1,080
  • 20
  • 44
  • From the this thread https://github.com/Azure/azure-service-bus-java/issues/144 "That is expected. The client traces everything. As you see, it doesn't cause any functional issues." The fact that service hang and didn't responds was not due to that trace but to our setting on the queue being to low. – Chris Jan 14 '20 at 20:29

2 Answers2

0

I had a similar problem and after some digging I foundthat the problem was network related.

I have done a test

  • start receiver (listening on topic in my case)
  • down network interface
  • wait some time (5 minutes)
  • up the network
  • send something on topic

In return I have got following

02:43:23.339 [reactor-executor-1] ERROR c.a.m.s.FluxAutoLockRenew$LockRenewSubscriber - Errors occurred upstream.
connectionId[MF_541d1c_1619568823389] sessionName[run-job/subscriptions/test-consumer] entityPath[run-job/subscriptions/test-consumer] linkName[run-job/subscriptions/test-consumer_432ec3_1619568824168] Cannot create receive link from a closed session.
02:43:23.344 [ioapp-compute-1] DEBUG com.pg.domain.airflow.RunDagPipeline - Received message: Left(ServiceBusError(com.azure.messaging.servicebus.ServiceBusException: connectionId[MF_541d1c_1619568823389] sessionName[run-job/subscriptions/test-consumer] entityPath[run-job/subscriptions/test-consumer] linkName[run-job/subscriptions/test-consumer_432ec3_1619568824168] Cannot create receive link from a closed session.,com.azure.messaging.servicebus.ServiceBusReceiverAsyncClient@46111612))
slavik
  • 1,223
  • 15
  • 17
-1

What you posted here is the trace, not the error. Yes, the service closes connections which are inactive for 10 minutes. The client traces it and reopens the connection. It is seamless, doesn't throw any exceptions to the application. That can't be your problem. If your sends are failing means there may be another problem, but not this one.

When you call sendAsync(), it returns a completable future. If there are no exceptions thrown to the application, that means everything is alright.

leopal
  • 4,711
  • 1
  • 25
  • 35
jaya surya
  • 1