-1


I am developing a P2P application which I am making it scalable to tens of millions of users. I am broadcasting a packet to each of these other peers and expecting a response. Before I go ahead with my coding I wanted to confirm if I can send a packet to 10s of millions of different IP addresses in less than a minute. and if they all respond back then will my application or even my PC be able to handle that many connections and packets in such small time ?

Using TCP and Windows.
Maximum CPU usage allowed : 20%
Internet bandwidth : assume 2Mbps
Application based on C programming using WinSock2
Assume a normal PC with 2 GB Ram, Core 2 Duo, 2.8 GHz

S. Swaroop
  • 411
  • 1
  • 6
  • 17
  • Create a quick mock-test. –  Jan 17 '13 at 18:38
  • Yeah, and how is that so ? Should I create 10s of millions of threads to send me packets simultaneously ? I don't think I own a super computer here ... Any better option ? – S. Swaroop Jan 17 '13 at 19:08
  • 2
    With all due respect, I don't think you're ready to develop such an application, let alone make it scalable to tens of millions of users. – Nik Bougalis Jan 17 '13 at 19:34
  • 1
    @S.Swaroop No, that would be silly. However, if a simple mock/stress framework of these functional requirements can't be written, then I suspect the scope of the actual program/protocol is currently out of grasp. (And it's *much* easier to write the mock program here.) –  Jan 17 '13 at 21:57
  • You can't broadcast with TCP, so your question already doesn't make sense. You need to investigate UDP multicast. – user207421 Jan 17 '13 at 22:30
  • Take a look at distributed hash tables. – Jan Wrobel Jan 18 '13 at 17:02
  • @JanWrobel: How distributed hash tables can help here? – Andriy Tylychko Jan 21 '13 at 14:37
  • can you please describe in high level your problem domain? I mean, what type of application you try to develop? – Andriy Tylychko Jan 21 '13 at 14:38

2 Answers2

1

Um, yes you can, but you might need to use UDP. However the responses backc will be a self DoS as well.

Ray
  • 40,256
  • 21
  • 101
  • 138
  • Yeah I was thinking the same but if I use UDP instead of TCP what benefit will I get? I mean we dont need to make connections and handshake and all but in the end I have to handle all the packets. Is there any other way to develop the program, any concept may be ? – S. Swaroop Jan 17 '13 at 20:00
1

Generally applications which need to directly communicate with tens of millions of users within the space of a minute will be based on clusters of servers that have the available internet bandwidth and computational power such that they don't DoS themselves. A P2P application shouldn't require a single host to communicate with that many users, especially not within that short of a time frame.

Ray is correct that, even if you were able to send the message, you will end up DoSing yourself with the response unless you put a lot of varied-length intentional delays in the client programs to space out their responses. He's also correct that you should use UDP if you were to attempt this. I find it very unlikely that your operating system supports maintaining 10,000,000 concurrent TCP connections.

In order to send a notification to tens of millions of hosts from a single host, the original host should notify some small subset of size n of the list of tens of millions of hosts. Each of those hosts would, in turn, notify n more hosts and so on. This would require on the order of n log_n_(total number of hosts) time vs. on the order of the number of hosts time.

If the response message is simply an acknowledgement of the receipt of the original message, a system opposite this can be used for the acknowledgements. Each host can send an ack to the one that sent it the message, then once that host has received all of the acks or a timeout occurs, it sends an ack to the host that sent it the message which includes the information of which hosts have already sent it an ack. This process continues back up the tree until it the combined acks reach the original host. This means you receive on the order of n responses back to the original host, not on the order of tens of millions.

If the response is more than just an ack, then your application is probably not scalable for anything remotely close to the hardware you describe as that's going to be way too much incoming data in too short of a time. Most likely you'll DoS yourself and quite possibly get a nastygram from your ISP.

reirab
  • 1,535
  • 14
  • 32