0

An End to End architecture is defined as follows:

JMS queue->Kakfa cluster->spark streaming->Hbase table

For Performance testing whole architecture, i can produce xmls through tools like Jmeter to JMS queue(will give response times of sending data to jms only) which flows through various components to Hbase.

I can even test each and every components individually(jms, kafka, spark),(which will give response times for particular block) I want to prepare script in jmeter which should inject data into JMS queue and get the response time for data reaching hbase.(end-end benchmarking)

Is this any way possible? Or am i going in wrong direction?

Any suggestions to achieve this? Suggestions will be appreciated.

Vishal Chepuri
  • 306
  • 5
  • 26
  • Are you trying to collect throughput or latency metrics because you can tune for better results for one or the other but not both at the same time? – Hans Jespersen Aug 26 '17 at 18:28
  • Why use a JMS broker and a Kafka broker? The throughput of the JMS broker will likely be much slower than Kafka so you end up benchmarking the slowest link in the chain. Perhaps try to use the Confluent JMS Kafka client so the publisher can send JMS straight into Kafka – Hans Jespersen Aug 26 '17 at 18:30
  • as we are getting data from a MDM(master data management) it wasn't possible for us to get data directly into Kafka. – Vishal Chepuri Aug 27 '17 at 09:37
  • @Hans Jesperen as i am working in perspective of Benchmarking. How can i achieve? – Vishal Chepuri Aug 27 '17 at 09:40
  • Put some time stamps in the JMS messages and make sure all the clocks are synchronized (using NTP) and you should be able to see the difference in time stamps between the message data and the create time in Hbase – Hans Jespersen Aug 27 '17 at 17:00
  • Thank you @HansJespersen your answer helped me for benchmarking . Can we do load test on same scenario(end-end)? In case of single transaction it will be easier to track time stamp, but what about load test? – Vishal Chepuri Aug 28 '17 at 16:17

0 Answers0