-1

This is a very basic question about the mechanics of computers. What are some of the common ways that a host handles multiple requests during the code execution phase for applications with millions of requests per second?

For example, suppose two users click a button to reserve a product on a website. Let's say the site takes 20 seconds to reserve the product in a database. User A's request hits the server first. But User B doesn't have to wait until A's request is finished right? Why is that? What is going on that enables multiple requests to be simultaneously processed? Is the server processing a little bit of each request and then switching to another request and doing a little bit of it, etc? Is the server starting a new process for each request that runs continuously?

I realize there could be different answers to this question, but what are some of the common ones.

user7340
  • 99
  • 2

2 Answers2

0

The basic theory is called multitasking. It's quite an involved subject. You can read up on it more here.

CB_Ron
  • 338
  • 2
  • 10
0

Here is a brief overview. I'm sure I'll get something wrong and that someone will point it out.

Servers such as Apache & Nginx are multi-threaded applications and will generally launch multiple threads on start up, often called Workers. Each thread will be capable of handling a number of connections, generally specified in the config file.

Whilst the workers are threaded the main http processes is generally single threaded and manages the threads it spawned. It does this because some operations are thread-safe whilst others are not. For example, writing to a log file is generally not thread safe. See this wiki for details on threading.

Basic example

If we take HTTP as an example;

Three people make a request for the same, or perhaps different pages. A worker answers the request, which comes in to the HTTP process listening port (generally port 80). The HTTP server processes the request and sends the response out on a different port, generally a high port such as 54501, or 34667. The reply and payload from the HTTP server do not get returned on port 80.

There is often a KeepAlive value configured within the HTTP server which keeps that connection alive until it times out, if there are no further communications. If further communication is received the worker is already primed and ready to go. If there are no more communications the worker closes the connection and it is either terminated or left idle listening for a new connection, depending on how it has been configured.

Apache, for example, may have the following;

<IfModule prefork.c>
   StartServers       4
   MinSpareServers    4
   MaxSpareServers   8
   ServerLimit      256
   MaxClients       256
   MaxRequestsPerChild  10000
</IfModule>

This basically tells Apache on startup to start 4 servers, have 4 in reserve in case they are needed and start a maximum of 256 with each servicing up to 256 connections.

Nginx on the other hand users "worker_connections"

You can also assign workers to specific CPU Cores & specify other fine tuning such as Priority etc.

How an operating system handles threads varies from one to another, for example, Linux does things differently from Windows.

Many servers are Linux or Unix based because they run without a GUI interface thereby dedicating all resources to what it was designed to do. A GUI consumes a lot of resources!

This has a very brief explanation and I hope it helps. I'm sure I got some of it wrong but it gives you an bit of an idea.

Admiral Noisy Bottom
  • 323
  • 1
  • 9