5

I'm a bit confused about how a server handles thousands of requests. I'd imagine that if there are thousands of requests, eventually two of them would have to hit the server at the exact same time. If two messages hit it at the same time, it can only handle one right?

I know that if multiple requests hit a server in very close, but not exact same, times then it can distribute the work to other servers very quickly before it gets another job to do. What I don't get, is what happens when a machine gets two requests at precisely the same moments.

Alxander
  • 323
  • 1
  • 3
  • 9
  • which server ...? btw all are multi threaded , so they are able to handle requests – Freak Jul 03 '13 at 03:34
  • How does a single computer do _anything_ at the same time? – Matt Ball Jul 03 '13 at 03:35
  • posible duplicate of http://stackoverflow.com/questions/8278887/how-does-a-server-handle-web-service-requests-from-multiple-clients – Freak Jul 03 '13 at 03:35
  • @Matt Ball The way I understand it, there is always a main thread that executes other threads which can run in parallel. However, the main thread can only get one input at a time. – Alxander Jul 03 '13 at 03:39

2 Answers2

9

Even if two cpu's are handling requests at exactly the same time, and they conflict, then at some stage they will say "I want to do something, which can't be done at the same time as anything else is happening". If two cpus do this at exactly the same time, there will be a predefined order - I would imagine that the lowered-number CPU would get priority.

This blocking will only take a cycle or two - just long enough to say "I lay claim to this resource" (known as a lock). The block will then be unblocked, and the other CPU can resume (so long as it doesn't want to access the same data).

This is hideously oversimplified.

Also, it is not possible for two requests to "come in at the same time" over the network, if there is only one network card - if they come in at the same time, EXACTLY (on a gigabit network), then the two packets will collide; both requests will be retried after a slightly random time, until there is no collision.

Remember, this is running at the clock speed of your computer (e.g. up to 3ghz), so "thousands per seconds" are as nothing. 1 million requests per second, would come out as 1 request per 1000 cpu cycles.

AMADANON Inc.
  • 5,753
  • 21
  • 31
  • Hmmm, I get that it would be extremely rare for a cpu to get requests at the exact same time, but what if it happens? When you say two cpu's, do you mean that something like a load balancer would just process a request on another cpu if one is already being used? What if both are busy? – Alxander Jul 03 '13 at 03:46
  • Thanks, that second last paragraph was what I was looking for. Just one more thing. Is the retry user side? – Alxander Jul 03 '13 at 03:48
  • I'm talking about a multi-core computer. One core can only receive one bit at a time. Requests can only ENTER one core at a time. If two requests come in two different network cards AT THE SAME TIME, then one will get priority (lower number on the bus?), and one request will edge forward. A single CPU can only run a single instruction at once. If you have a multi-core, and there is contention between the cores, one will have priority. This is the time it takes for an electrical signal to go from a 0 to a 1 , which is a lot faster than the CPU even. – AMADANON Inc. Jul 03 '13 at 03:53
  • If a packet comes in to a network card, a signal (called an interrupt) is sent to a cpu, and the data will be held there until a cpu is available to pick it up. – AMADANON Inc. Jul 03 '13 at 03:56
  • Alxander: In TCP (which is what web pages use), every data packet will get an acknowledgement; if the sender does not receive the acknowledgement, they will resend after a while - typically 3-5 seconds from memory. For any network traffic that does not use TCP (e.g. skype), it is up to the sender and receiver to work it out between them - in the case of skype, there's no point getting a lost packet, because it'll already be too late; the lost data is ignored, and you will hear a small drop-out on your call. – AMADANON Inc. Jul 03 '13 at 04:01
0

All servers are possibly concurrrent Servers.The server can be iterative, i.e. it iterates through each client and serves one request at a time. Alternatively, a server can handle multiple clients at the same time in parallel, and this type of a server is called a concurrent server.
Also they are generating response by load balancing and making web cache.
These two important mechanisms are used to allow many users at the same time.
Web Cache
A web cache is a mechanism for the temporary storage (caching) of web documents, such as HTML pages and images, to reduce bandwidth usage, server load, and perceived lag. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.1 Google's cache link in its search results provides a way of retrieving information from websites that have recently gone down and a way of retrieving data more quickly than by clicking the direct link.From Wiki
Load balancing
Load balancing is a computer networking method for distributing workloads across multiple computers or a computer cluster, network links, central processing units, disk drives, or other resources. Successful load balancing optimizes resource use, maximizes throughput, minimizes response time, and avoids overload. Using multiple components with load balancing instead of a single component may increase reliability through redundancy. Load balancing is usually provided by dedicated software or hardware, such as a multilayer switch or a Domain Name System server Process.From Wiki

This is a little picture of multi threaded server

Freak
  • 6,786
  • 5
  • 36
  • 54
  • But what happens if the load balancer itself gets two simultaneous requests? Not just ones in very close proximity. – Alxander Jul 03 '13 at 03:40
  • All load balancers are capable of making traffic decisions based on traditional OSI layer 2 and 3 information. More advanced load balancers can make intelligent traffic management decisions based on specific layer 4–7 information contained within the request issued by the client.Load balancing decisions are made quickly, usually in less than one millisecond, and high-performance load balancers can make millions of decisions per second.see http://www.citrix.com/glossary/load-balancing.html – Freak Jul 03 '13 at 03:49