0

Ok, say I have simple web-server instance which always responses "Hello world" and this simple server can handle only 1K connections simultaneously.

I have to handle 1M http-requests simultaneously, so I clone 1K nodes with my web-server instance.

But I have only one entry point. All requests come through one point.

So what should be placed as entry point to delegate requests? What software can pipe 1M requests to 1K web-servers?

corvax
  • 1,095
  • 1
  • 10
  • 35

1 Answers1

6

If you use round-robin DNS, the requests can come through multiple points.

If you want software for reverse proxy on Linux, Nginx is the way to go.

node-http-proxy also seems quite popular.

Golang has a ReverseProxy built into its httputil package.

(If I'm not mistaken, all three above will use epoll on Linux. But there is no guarantee any software can always handle 1M connections. It depends on many factors.)

Load balancing is a broad topic. Here are a few links for reading:

https://en.wikipedia.org/wiki/Round-robin_DNS

https://en.wikipedia.org/wiki/C10k_problem

https://en.wikipedia.org/wiki/Clustered_web_hosting

https://en.wikipedia.org/wiki/Reverse_proxy

https://en.wikipedia.org/wiki/Load_balancing_(computing)

cshu
  • 5,654
  • 28
  • 44
  • thank you! I know a lot of reverse proxy instruments, but non of them answers question how to design architecture to handle 1M requests same time. Links you provided don't explain how to do that. – corvax Dec 27 '16 at 08:29
  • @corvax http://blog.charmes.net/2015/07/reverse-proxy-in-go.html Here you can find an example. The author wrote a `NewMultipleHostReverseProxy` to wrap `ReverseProxy` and use multiple hosts. When a request comes in, a host is selected randomly. – cshu Dec 27 '16 at 08:54
  • so you say, this reverse proxy written in go will receive 1M requests, will open 1M connections and will wait for an answer from worker nodes? – corvax Dec 27 '16 at 08:58
  • The question is not about load balancing, the question is about processing 1M requests same time. Any load balancer can only open limited number of connections, I need 1M. – corvax Dec 27 '16 at 09:00
  • @corvax A google shows how some people did 1M. But it depends on a lot of factors like hardware, config, etc. Load balancer is the entry point, it direct requests to multiple worker nodes. – cshu Dec 27 '16 at 09:10
  • Every question could be answered with following phrase "Google shows...". Are you sure you understand how load balancer work? Try to create stress test over 100K requests to any load balancer. It always creates socket and waits for an answer, number of sockets are limited, so your points to load balancing without explaining "how to handle 1M (!!!) requests" are useless. – corvax Dec 27 '16 at 10:29
  • @corvax Of course it's limited... I don't know what answer you expected to get. If you want a step-by-step guide about everything in order to pass the 1M stress test, then I guess my answers is indeed as good as useless. – cshu Dec 27 '16 at 11:51
  • It's better not to give an answer instead of writing useless things. "Google shows..." and "nginx, nginx, NGINX!!!!". – corvax Dec 27 '16 at 16:25
  • @corvax to handle that much load (1M conc) and with the knowledge how much an instance of your application server can handle, you have to learn that number for your load balancer also. Let's say each one can handle 100K so you need 10 of them, then you have to balance them to reach your million. A way could be with DNS as cshu suggested, there are others too. Usually there isn't one size fits all in these situation you have to build the infrastructure that suits best in your needs. For example what is the size of your requests in these numbers you have to deal with bandwidth problems also. – zochamx Dec 29 '16 at 12:45