How to enable multi threading in Camel quickfixj component

Question

I am using camel quickfix component for consuming market prices for different currency pair. I am subscribing for around G20 currency pair so we are getting lot's of update and our ennd point is not able to handle such load so is start rejecting message and logs error .

Sending time accuracy problem

I am thinking it to make multi threaded so more then one thread can handle prices updated. I tried to search a lot but didn't find any satisfactory answer.

Can you please help me out in this ?

After fetching market prices what exacty you are doing i.e any operation or sending to queue or some component? quickfix is not able to handle or some other component? Also, share you error. — Pratiyush Kumar Singh, Apr 28 '16 at 08:51

score 3 · Answer 1 · answered Apr 25 '16 at 16:01

3

There is 3 ways to do multi threading with Camel :

SEDA route : launch a route in a new thread
JMS/ActiveMQ for parallel processing
JPA : use database as message broker

An example with an activemq solution :

<route> // quikfix endpoint route
  <from uri="quickfix-server:META-INF/quickfix/server.cfg"/> // QuickFix engine who will receive the message from FIX gateway
  <to uri="uri="activemq:queue:fix"/>"
</route>

<route> // parralize route
  <from uri="activemq:queue:fix"/>
  <bean ref="fixService" method="treatment"/> // do your stuff
</route>

answered Apr 25 '16 at 16:01

Thomas

1,008
3
16
34

Thanks for answer but still we would be consuming market data in single thread . Does publishing on active my would not hit performance ? – user1047873 Apr 26 '16 at 04:56
This is the aim of activemq and it's very lightweight. You can also choose an another implementation of JMS to fit your needs. You can consume data in the same thread but process it in different threads launch with jms or seda route. – Thomas Apr 26 '16 at 07:45

score 2 · Answer 2 · answered Apr 27 '16 at 09:06

You may use Threads DSL ("Using the Threads DSL"). Example:

<bean id="threadPool" class="java.util.concurrent.Executors" factory-method="newFixedThreadPool">       
    <constructor-arg index="0" value="20"/>
</bean>

<camelContext id="myContext" xmlns="http://camel.apache.org/schema/spring">

    <route> 
      <from uri="quickfix-server:META-INF/quickfix/server.cfg"/>
      <threads executorServiceRef="threadPool">
         <process ref="someProcessor"/>"
         ....
         other logic that should be run in concurrent environment
         .... 
     </threads> 
    </route>

</camelContext>

As you see, you can use thread pools from java.util.concurrent package. Another option is that you can set threads number directly:

    <route> 
      <from uri="quickfix-server:META-INF/quickfix/server.cfg"/>
      <threads maxPoolSize="20">
         <process ref="someProcessor"/>"
         ....
         other logic that should be run in concurrent environment
     </threads> 
    </route>

</camelContext>

score 2 · Accepted Answer · answered Apr 27 '16 at 18:52

issue: SendingTime accuracy problem

Root Cause Analysis:

The above error message is usually followed by a session logout. It is caused due to incorrect date and time settings in the client machine.

Solution:

Verify date, time and time zone are all set to the correct date and time settings. Since you can set time zone and time independently, I suggest to double check the time zone (UTC hours difference), matches the time set.

There is a CheckLatency and MaxLatency config option, see http://www.quickfixengine.org/quickfix/doc/html/configuration.html#Validation

You can use two config options to modify the behavior in relation to time synchronization issues:

This option turns the latency check on or off:

CheckLatency=[Y|N]

This option tunes the maximum latency difference (120 seconds is the default): MaxLatency=120 or >120

There are another 2 ways to solve the issue.

First, this problem can be avoided by

logging out the client before reset the session,
changing from day session to weekly session

Second, this problem can be solved by clearing up the queued messages.

Resource Link:

thanks for you answer . this issue was because my system was not able to process all messages on time. Anyway this way i am not getting same message. — user1047873, Apr 28 '16 at 10:07