0

I am using Active MQ storage mechanism for my UDP packet processing. One UDP packet processing Program is listening UDP packets and inserting into 4 Queues which is configured. I have one consumer program which is consuming from these 4 queues by creating 4 threads. Each thread process the packet and insert into db and then call a webservice to do various functionalists and then only process the next packet. My problem is in some days when the frequency of getting packets are very high, the consumer is not consuming fastly and Queuing is happening. However in Queuing time and Non-Queuing Time the webservice processing time for a packet is same and almost constant to 50 ms.

So during this Quing time I will have to stop the webservice and restore the issue. When I am stoping the webservice the Dequeuing is happening fastly. But I want to use all the functionalities of webservice and the processing should be also high. Because of this Queuing Issue Recently I have increased the number of Qs from 4 to 6 and for some time it restored my issue. But as and when the number of packets are getting high Queuing is happening and the processing is not happening in higher rate.

Please advice how can I solve the issue ?

Thanks for you time.

user3177635
  • 31
  • 1
  • 2
  • I fail to fully understand the problem: are the consumers to slow in processing the messages sent to them? or is the ActiveMQ broker not fast enough? What is the problem with messages queuing up? Do you need to guarantee 50ms processing time? Have you ruled out the database as the bottle neck? – SirRichie Jan 09 '14 at 22:08
  • I have 6 Qs and correspondingly 6 threads of a single consumer program is consuming from the Qs.(One thread for one Q). The minimum average processing time required is 150 ms for one thread. I can't decrease the processing time less than an extend. So how can I solve this issue ? will Increasing the number of threads (no of Qs) help ? It is i3 ubuntu machiene. – user3177635 Jan 20 '14 at 05:19
  • If all threads run on the same machine and you want to decrease processing time, the only setting in which increasing the number of threads would help is, if the threads were waiting at some point during processing. I don't think this is the case for you. – SirRichie Jan 20 '14 at 09:12
  • Yes.There is a webservice call for every thread which will take some time. – user3177635 Jan 22 '14 at 10:56
  • So your bottleneck is the webservice call, not the number of threads that consume messages. You'll have to try making that call faster. – SirRichie Jan 22 '14 at 16:14
  • Thanks for your comments..But I can't make it more faster. Some processes should be done via webservice call which will take a constant time to process – user3177635 Jan 24 '14 at 04:26
  • I'm afraid you're out of luck then – SirRichie Jan 24 '14 at 07:43

0 Answers0