0

I need to complete some performance tests on SOA appliances with side cache. I have developed a simple application to generate SOAP/HTTP traffic but now i need some way of monitoring the E2E applications performance.

One vital metric i require is an accurate figure for the Transactions Per Second, as well as e2e response time.

I've used soapUI and Loadui but just do not believe the reported TPS figures as they seem very high, e.g. > 1300 TPS.

can anyone recommend a method to measure TPS that is "fool proof"?

Hector
  • 4,016
  • 21
  • 112
  • 211

1 Answers1

1

I'd suggest cross checking SoapUI's numbers against the logs from your server (count the number of lines with the same second), or cross check this way:

Time the test run yourself.
Verify that the number of transactions SoapUI cites are accurate (logs or another measure on the server itself). Divide trans count by seconds.

In the past I've done this and found SoapUI to be pretty reliable.

One thing to keep in mind in terms of whether your numbers are as good as they can be is whether or not you might need to simultaneously run soapui from more than one machine. I suggest monitoring the CPU, memory, bandwidth, etc on the SoapUI machine. If any of these get rather high, run the test on two machines simultaneously with very close to the same start and stop times and then you can safely add the two TPS numbers.

chrismead
  • 2,163
  • 3
  • 24
  • 36
  • thanks for your answer. i will put some logging on my server to assist in calculating my tps. i did try running two client machines. while one soapui quoted 1300 tps, the second just 800 tps. – Hector Jul 14 '12 at 07:25
  • Yeah, I see that type of thing, too, sometimes. Was the one that got up to 1300 started slightly before the other one? – chrismead Jul 14 '12 at 17:49
  • That is pretty normal, then, at least that is my guess. The first one seems to get more server resources and keep them throughout the run. – chrismead Jul 14 '12 at 19:22