How Do I Analyse Results of Load Testing to Determine Hardware Requirements

Question

When running load tests for an application server, the average results are as follows for 1.25 concurrent users.

I calculated 1.25 concurrent users by taking a peak hour of visitors (5 visitors) for a 15 minute duration (900 seconds), then divided by 3600.

Average Result:

Completed Requests: 2683
Requests Per Second: 134
Mean Latency: 14.8ms

This is on a server with 1GB of RAM and 1 vCPU. The CPU usage didn't go above 30%. Memory seemed unaffected.

So what exactly does it mean to have a result of 134 Requests Per Second? I'm trying to determine the server requirements that I'll require to handle X users and how this will scale, but I'm having trouble finding information on the correlation between concurrent users and request per seconds in terms of scalability.

Anything to help me further understand how to analyse these results would be very helpful, thank you.

Answer 1

Your calculations look a little bit weird, maybe your application is very exotic, however normally 1 user doesn't perform 100+ requests per second, more likely it would be something like 1 request per several seconds + X requests to download embedded resources (images, scripts, styles, fonts) and eventually AJAX requests. Check out What is the Relationship Between Users and Hits Per Second? article for example real life use case.

There are 2 main types of performance testing:

1. Load Testing - when you put your system under anticipated load, i.e. 5 users and see whether it is capable to provide reasonably low response time
2. Stress Testing - when you're trying to identify the breaking point of your system or the first bottleneck by starting with 1 virtual user and gradually increasing the load until response time exceeds acceptable threshold or errors start occurring (whatever comes the first)

So first of all ensure that your load test 100% matches the network footprint which is produced by the real user using the real browser in terms of cookies, headers, cache, think times, etc.

Once done you can run the load test against your system to see whether it works find for the expected number of users.

And finally you can increase the number of users till anticipated maximum or when response time becomes too high or you start seeing errors.

Answer 2

Those two measurements are not universally related in the same way. You have to estimate that for your website: One website may require 2 requests, per user, per day. Another website may require 200+ request, per user, per minute.

For a rough estimate, open your favorite browsers web development tools, watch how many requests a typical page open consists of, make an educated guess how many pages are opened by any unique user per minute. Then multiply.

Example:

• 25 concurrent users during peak times
• 3 pages navigated by each user per minute
• 2 requests (1 html, 1 css) per page opened (upper bound, caching reduces actual number)
• 25 * 3/min * 2 requests = 150 requests per minute

My server can complete about 150 requests per second, so I do not expect any significant delay for 150 per minute. This is 60x difference, so as I approach 60 times my current user peak, I will have to expect increased load times or even failures.

Do not expect such calculation to give you more than a very rough idea of what kind of userbase you will be able to serve concurrently:

1. Some requests may place many orders of magnitude more stress on your limited server resources, so counting requests cannot fully account for actual load. If necessary, compare dynamic (wsgi, fcgi) requests separately from static content (css, images, js)
2. If you make your calculation based on actual users, you are discarding background stress placed on your server by Google/Shodan/Vulnerability Scanners/Botnets/.. - for smaller websites, that may be a significant share of your server load.