1

I want to implement an ajax Client that does polling to the server. This leads to millions of small requests...

Do you have any rough estimates (based on your hardware and your experiences, I do currenlty not have any dedicated server hardware yet), how man requets a sever can handle for example Tomcat 7 with standard server hardware (8GB Ram, 4 Cores, 2,5GHZ each)? The internal Processing of one request is estemated to finish within 50 millisecons (only data is put into a Ram Cache, some counters are incrementd, light textprocessing and then data is read from memory again to return to the client. it will all fit in RAM).

I would be thankfull for any experiences you made how much requests you think or you were able to handle on your server in a comparable environment.

thanks!! jens

jens
  • 16,455
  • 4
  • 21
  • 20
  • 1
    the answer will be dependent on the CPU time your application uses for serving a request. if you are at 100% CPU for serving a request during those 50ms then you will be confined to 4x1000/50 = 80req/sec at the most – Asaf Apr 12 '11 at 06:01
  • I would try profiling your server under a realistic load as 50 ms sounds like a long time to me. There must be some performance improvements you can make. BTW: I don't know about tomcat, but you can do something similar without tomcat for a persistent connection in a one hundredth of the time or less. – Peter Lawrey Apr 12 '11 at 06:43

1 Answers1

1

50ms gives you quite a lot of processing, actually... especially if most of these requests are just polling, presumably for changes. How did you come up with that estimate? That would keep 4 cores busy with 80 requests per second, of course... but that's not an awful lot, and you probably wouldn't want to run your servers at full capacity the whole time, and there'll be some overhead for the simple handling of the networking.

To be honest, estimates are relatively pointless compared with just trying it. You'll need to load test your servers anyway, so it would be best to do that now rather than later.

Note that an alternative to frequent polling would be to have long "hanging" requests, aka Comet. Each request would be held unfulfilled for some period (e.g. 5-10 minutes) and as soon as you had a change, you'd respond appropriately. That way you can drastically reduce the number of requests - but it does make the code much harder to write (as it has to be asynchronous; you don't want one thread per request at that point).

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • FYI it is best to have a comet-like long poll of no longer than 30 seconds, because many residential routers and firewalls will discard a TCP connection that is idle for more than a certain amount of time, and empirically that appears to be around 30 seconds. If it is discarded and the server tries to send data on that poll it will get a RST. – Jim Morris Apr 12 '11 at 06:06
  • thanks, comet is a really good startingpoint to search for more. it seems that tomcat also supports comet. I will research further on this, thanks! – jens Apr 12 '11 at 06:10
  • @Jim: Hmm... that's not my experience based on working with a sync product. For example, I believe the iPhone will typically hold requests to Exchange servers open for 5 minutes or more. – Jon Skeet Apr 12 '11 at 06:13
  • @Jon: It depends on what routers it goes through, a typical residential NAT router will only keep the NAT info around for a short while, this can be configured in some but the default is usually in the order of minutes. In my experience I found that the shortest time is 30 seconds if you want it to run everywhere. I suspect IPhone is not going through a NAT router or a firewall for that matter. – Jim Morris Apr 12 '11 at 06:16
  • @Jim: Again, not my experience... and if this is such a problem, why would Apple use 5 minutes as the default? (I think it may actually be slightly longer than 5 minutes - I can't remember exactly, but it's definitely on that order; much longer than 30 seconds.) Of course it would be worth the client trying to adjust for such situations if it can easily do so. – Jon Skeet Apr 12 '11 at 06:19
  • @Jim: To respond to the edit in your comment - the iPhone *will* use a wireless network if it's available, at which point it's going through exactly the same route as anything else on that network, such as a PC. However, it feels like there ought to be some data around this. I'll look around for articles - do you know of any? – Jon Skeet Apr 12 '11 at 06:20
  • @Jon: I'd be happy to discuss it with you, I have a huge amount of experience in this area, but it is slightly off topic here so message me or email me. – Jim Morris Apr 12 '11 at 06:25
  • @Jim: I'm not *that* bothered personally - but perhaps the OP would want to get in touch. If you can't hold the connection open for longer than 30 seconds, it reduces a lot of the benefits of Comet from a resources point of view :( (You still get "instant" responses of course.) – Jon Skeet Apr 12 '11 at 06:26
  • @Jim: http://stackoverflow.com/questions/1338123/what-is-a-safe-amount-of-time-that-i-can-wait-before-responding-to-a-browser-wit suggests 30 seconds too, but due to the browser rather than the router. That would explain how the iPhone gets away with it, of course - because they can make sure it doesn't get dropped from the *client* side at least. Of course, it's possible that's just *another* issue... – Jon Skeet Apr 12 '11 at 06:29
  • @Jon: If the TCP connection has keepalive set, then it will keep the TCP connection alive. maybe IPhone sets keepalive, in which case there is no limit. However that is not the default for most systems. You can easily test this using nc to a server you have access to, and don't type anything for 2 minutes, you will find the connection has gone away. I know my firewall holds NAT and firewall holes open for 45 seconds. I found the 30 seconds empirically while running a 3 million user voice service. I don't think there is any definitive answer which is why I opt for the failsafe of 30 secs. – Jim Morris Apr 12 '11 at 06:38