Back of envelope calculations: Imagine that I have some requirements and after some calculations, I get that I will get about 100k request per second to my server. How many servers would I need? Let’s say server has 32CPUs and each request takes 100ms
Thanks
It depends on the type of request and the system architecture . You also need to do few number of iterative stress tests, profiling tests for measuring various parameters of your system for various use cases in hand to see if all performance metrics are within your expected limit for more appropriate realistic data.
If the request is CPU bound , the generic formula as below can be used :
Max number of requests per second = Number of CPU Cores / Average request(task) time(seconds)
If you have a server with 32 CPU cores and if every task consumes 100 ms then, then you can expect the CPU to handle approximately 32 CPU cores / 0.1 seconds = 320 requests per second.
If the request is memory bound, the generic formula as below can be used:
Max number of requests per second = (Total RAM / worker memory) * (1 / task time)
If the total RAM memory is 16Gb and the worker/process memory consumption is 40Mb and the task consumes 100ms, then the maximum number of request that you can expect the CPU can handle shall be 4000 requests per second.
You can use validation tools like Apache Jmeter that has provisions for performing simulation of steps, plugins for RPS generation like Throughput Shaping Timer and dashboard view for analysis and decision making.
You need to make a performance test with a single server to see how many RPS it supports, then you're going to have the answer. You also need to consider adding a Load Balancer in front of your servers, which dispatches the requests to all the servers (in a round-robin way, maybe).
