12

I'm reading about Channels in the JDK 7 docs (here), and stumbled upon this:

Multiplexed, non-blocking I/O, which is much more scalable than thread-oriented, blocking I/O, [...]

Is there a simple explanation as to why this is so?

Mads Nielsen
  • 634
  • 5
  • 16

2 Answers2

6

Because a thread stack is usually much larger than the data structure needed to support an async I/O connection. Also, scheduling thousands of threads is inefficient.

Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • 3
    That's the usual argument, but: I can run tens of thousands of threads on my desktop machine, but I've yet to see any problem where I could actually serve tens of thousands of connections from a single machine without everything crawling to a halt. So that seems a weak argument to me. Also NIO nonblocking with selectors [isn't such a great choice often enough](http://www.mailinator.com/tymaPaulMultithreaded.pdf) – Voo Jan 29 '12 at 23:16
  • @Voo: Some connections may be idle for tens of minutes at a time, but still open. – Ben Voigt Jan 29 '12 at 23:30
  • 1
    @Voo, I doubt you do have tens of thousands Runnable (not just idling) threads. context switch even cheap is still a TLB cache evict w/ more cache misses. Working myself on thousands of connections I'd say that serving 2k users w/ around 85k messages/second on dual socket xeon takes like 20% CPU (of all CPUS)... Also NIO allows for 'fair' traffic delivery which is very important and very often overlooked as it ensures stable latency for the clients. (Iirc the benchmark doesn't measure latency) – bestsss Jan 30 '12 at 13:44
  • @bestsss As I said, everything crawls to a halt way before the threshold if the threads do serious work. To clarify: The threads are doing some rather heavyweight work, so we reach the capacity of a single server before context switching overheads get a problem. You're right I only tested bandwidth (more important for my problems) and I don't think I've seen anything about latency so far. Would make for an interesting benchmark - will have to test that some day (also testing blocking NIO - have heard some good things about that) – Voo Jan 30 '12 at 16:08
  • @voo, using multiple threads you cant ensure fair traffic since it depends heavily on the OS scheduler and the network speed of the connected clients. – bestsss Jan 30 '12 at 16:58
3

"Blocking" means that threads have to wait as long as necessary for a resource to become available...which means, by definition, threads will be sitting around waiting for resources. Non-blocking avoids this sort of thing.

Generally, non-blocking solutions are trickier, but they avoid resource contention, which makes it much easier to scale up. (That said, the point of Channel is to make this less tricky.)

Louis Wasserman
  • 191,574
  • 25
  • 345
  • 413
  • 1
    Non-blocking I/O doesn't mean the threads don't wait, it simply means that the wait operation is separate from the I/O operation. – Ben Voigt Jan 30 '12 at 18:03