-2

I'm creating a REST API that will listen on a public URL and accept uploads of large amounts of data. I understand how to scale software processes using message queues, but what I don't understand is how to avoid a bottleneck between users and reaching my message queue server farm. To my understanding, at some point I have a machine listening to requests on an IP address, and that will be a single point of failure bottleneck. But this sounds really wrong since, you know, sites like google.com exist.

My subsequent question is if you can achieve this scalability for a fixed IP address instead of a fixed URL. Not because I have to do this, I just want to understand if in the practical sense of running a website (or other HTTP server) scalability can only be achieved by using a dynamic number of IPs.

djechlin
  • 95
  • 7
  • I'm not sure this question makes much sense as written; there are no limits on throughput built into IP. – MadHatter Jan 16 '14 at 15:21
  • @MadHatter in this universe, not per the IP protocol specification. – djechlin Jan 16 '14 at 15:22
  • This universe or otherwise, the question still doesn't make much sense. – ceejayoz Jan 16 '14 at 15:26
  • @MadHatter asked question from a different vantage point that hopefully makes more sense. – djechlin Jan 16 '14 at 15:27
  • @ceejayoz see rewrite. – djechlin Jan 16 '14 at 15:28
  • "To my understanding, at some point I have a machine listening to requests on an IP address, and that will be a single point of failure bottleneck. But this sounds really wrong since, you know, sites like google.com exist." http://en.wikipedia.org/wiki/Round-robin_DNS http://en.wikipedia.org/wiki/Multicast http://en.wikipedia.org/wiki/Geocast etc. – ceejayoz Jan 16 '14 at 15:28
  • @ceejayoz yes, that answers my question. If you post as answer (with a sentence of explanation) I'll accept. – djechlin Jan 16 '14 at 15:35
  • @ceejayoz in other words: I got downvotes, and completely rewrote my question, as you're supposed to do. You were able to easily answer my new question from here. Still downvotes though. Is there anything else I should be doing I'm not aware of? – djechlin Jan 16 '14 at 16:01
  • Well, even rewritten, it's not really on-topic here. – ceejayoz Jan 16 '14 at 16:08
  • @ceejayoz covered under "Network routing, switches, and firewalls". – djechlin Jan 16 '14 at 16:24
  • It's also essentially a request for learning materials, which is one of the close reasons. – ceejayoz Jan 16 '14 at 16:26
  • @ceejayoz at no point did I request learning materials, and I do not even see your basis for that. – djechlin Jan 16 '14 at 16:28

1 Answers1

2

What physical bottleneck determines how much data can be uploaded to one IP address?

The hop with the least bandwidth available for your traffic.

Updated question, IP Endpoint redundancy: See HSRP or CARP. See also Load Balancing

For multi-site redundnacy see: Anycast or Geocast, both of which are quite expensive and complicated.

Chris S
  • 77,945
  • 11
  • 124
  • 216