0

A vast difference between the results of performing load testing for 50 virtual users:

Method 1) Single Linux machine (m3.medium)

No of Threads: 50 users
Ramp Uptime: 60 sec

Method 2) JMeter primary-secondary distributed testing

i) server - linux machine (m3.medium)
ii) client1 - linux machine (m3.medium) - No of Threads: 25, Ramp up time: 60 sec
iii) client2 - linux machine (m3.medium) - No of Threads: 25, Ramp up time: 60 sec

jMeter version: 4.0 in each case

Distributed Testing takes about four times more time than single testing. I couldn't find the cause even after several test runs.

enter image description here

Shashank Shah
  • 31
  • 6
  • Are you sure all those source requests are actually hitting the target machine? 50 threads seems like it might be a lot. What happens if you try 24 threads on single machine, and then 12 and 12 on distributed? Or 10 and then 5 and 5? – Jonathan Jun 28 '18 at 18:33
  • Do all have the same amount of ram and java heap space? Are they located in the same subnet? – joCha Jun 28 '18 at 18:50
  • "takes about 4 times more time than single testing" - based on what? can you share actual example of what ran, and took different time on both environments? – timbre timbre Jun 28 '18 at 21:11
  • All the requests are hitting the target machine. All machines are in same subnet as well. Enough ram and heap is also available. – Shashank Shah Jun 29 '18 at 05:48
  • @kiril I mean while on single machine APIs which takes about few hundred milliseconds takes about 4x milliseconds when ran on distributed env. – Shashank Shah Jun 29 '18 at 06:01
  • @KirilS. I have attached a snapshot for your reference. – Shashank Shah Jun 29 '18 at 06:21
  • I think it's a result of too few samples. First of all 50 samples are not enough to make any conclusions to begin with (you need at least 1000, better 10k or more). Especially since JMeter doesn't have a concept of proper warmup. Second, when running 50 iterations on one machine, it's *not the same as* running 25+25 on 2 machines. IMagine 1st transaction is slow. So it matters if it's 1 in 25 or 1 in 50. But truth is it should not matter at all: if you run 10k transactions individual performance of 1 transaction won't matter, and this is what you are looking for – timbre timbre Jun 29 '18 at 19:44
  • Try one more approach and see if you're able to achieve the same result as #1. No of Threads: 25, Ramp up time: 60 sec With Two Machines at the same time on non distributed mode. I suspect Jmeter Server/Master machine is adding time while handling distributed testing with multiple clients/slaves. Also try to monitor response time on your Web Server, they may be different than Jmeter response time. – Yugal Jul 02 '18 at 10:12
  • @Yugal, jonathan - I tried different scenarios with lowering the sample counts but still could observe a distributed testing taking lot more time than single machine. I wasnt able to monitor data on web server. I will see how I can get aggregated data from nginx server on aws. – Shashank Shah Jul 02 '18 at 14:25
  • @KirilS. I am trying to use distributed testing so that I can test 1000 and eventually 10000 concurrent users. But with just 50 concurrent users, distributed testing is taking a lot of time in several APIs so it doesn't make sense to increase the users. Nevertheless I will still give it a try by increasing few hundred users. Also 1 in 25 is taking more time than 1 in 50 which is hard to digest. – Shashank Shah Jul 02 '18 at 14:33
  • I didn't say "add more users", I said "add more iterations": you need bigger sample data to get it more representative... – timbre timbre Jul 02 '18 at 20:21

1 Answers1

0

How many machines is needed for 50 concurrent users ? TL;DR 1

Use Distributed Testing only when 1 machine isn't enough

Use Distributed Testing only when have to, setting up has its overhead and tuning.

It's relevant for 1K concurrent users and above

Ori Marko
  • 56,308
  • 23
  • 131
  • 233
  • I am planning to load test up to 1000 concurrent users hence trying it out with lesser users initially. – Shashank Shah Jun 29 '18 at 05:46
  • @Shashank Shah well, that will be entirely different question,beacause you won't be able to execute 1K in single machine – Ori Marko Jul 04 '18 at 04:29