Apache Camel route timing out

Question

I have two Camel routes, configured in XML and pasted below: -

Route 1:

<camel:route id="statementsArchivingPollRoute">
    <camel:from uri="timer://tempQueue?fixedRate=true&amp;period=30s&amp;delay=30s"/>
    <camel:transacted ref="PROPAGATION_REQUIRED">
    <camel:process ref="statementsArchivingRequestZipProcessor"/>
    <camel:choice>
        <camel:when>
            <camel:simple>${body.size} >= 1</camel:simple>
            <camel:split>
                <camel:simple>${body}</camel:simple>
                <camel:marshal ref="archiveFileInterfaceMetadataMapper"/>
                <camel:to pattern="InOnly"
                          uri="activemq:{{ccs.activemq.queue.prefix}}.sr.archive.bulk.ingestion.req?jmsMessageType=Text"/>
            </camel:split>
            <camel:log loggingLevel="INFO" message="Archiving content was processed"/>
        </camel:when>
        <camel:otherwise>
            <camel:log loggingLevel="INFO" message="No archiving content to process"/>
        </camel:otherwise>
    </camel:choice>
    </camel:transacted>
</camel:route>

Route 2:

<camel:route id="statementsArchivingBulkIngestionRequestRoute">
    <camel:from uri="activemq:{{ccs.activemq.queue.prefix}}.sr.archive.bulk.ingestion.req"/>
    <camel:throttle timePeriodMillis="4000">
    <camel:constant>1</camel:constant>
    <camel:setExchangePattern pattern="InOnly"/>
    <camel:unmarshal ref="archiveFileInterfaceMetadataMapper"/>
    <camel:bean ref="archiveFileEntryTransformer" method="transform"/>
    <camel:setHeader headerName="CamelHttpMethod">
        <camel:constant>POST</camel:constant>
    </camel:setHeader>
    <camel:toD uri="{{ccs.bulk.ingestion.service.ingest.archive.file}}"/>
    </camel:throttle>
</camel:route>

The processor in the first route returns a list of request objects. The list is then split and each request is marshalled and placed on a queue.

The second route listens to this queue. When it de-queues a message, it unmarshals it, performs a transform and then uses it to send a post request to another service. I am throttling this route so that it only processes one message per second so as not to overwhelm the downstream service.

This all works fine when the list only contains a few requests and hence only a few messages enter the queue, but when there are many items in the list, route 2 times out and the log entry below appears:

Atomikos:12] c.a.icatch.imp.ActiveStateHandler        : Timeout/setRollbackOnly of ACTIVE coordinator !

The timeout leads to the process repeating itself and the downstream service ends up getting called multiple times per message instead of just once.

I can't understand why the number of times route 2 is invoked should cause it to timeout. I thought that a single instance of the route would be launched for every message de-queued from the activemq. If a single message takes a long time to complete, then I would understand, but clearly the timeout is based on the cumulative times of all messages being de-queued.

I am fairly new to Camel and I have clearly misunderstood something from an architectural point of view. I would be extremely grateful for any guidance into how to stop these timeouts occurring. Thank you for reading.

Answer 1

Jon, you may want to try following to investigate

1. disable/comment the 2nd route. Purpose of using activemq would be to make process async which means 1st route should not have any impact due to 2nd route. If this works without route 2 then problem is elsewhere.

2. If you find 1st route is working fine without 2nd route then next would be to try setting fewer number of threads in 2nd route. May be put 1 or 2 threads and see if that helps. I am thinking it is contention on activemq rather than these route configurations.

3. Check the payload size that you are pushing to activemq. If you are publishing very large message to activemq that may also have an impact as large number of items and each item is large causing contention in activemq and transaction is taking longer than the timeout setting.

If you are pushing large data set to activemq, you may want to revisit the design. Save payload in some persistence area (db/file/cache) and sent notification events containing only reference to the payload and some metadata. 2nd route can then take the reference from event and retrieve the payload from where it was saved by route 1.