6

I am new to Java NIO and have used it a little. I have a general query. If you are designing a ultra low latency app vs high throughput application, which of the two gets clearly benefited by using Non blocking IO?

My understanding is that Non blocking IO should certainly help in high throughput as worker threads are not blocking, hence not waiting for response and are free to fire new requests until previous requests are served. Once we get responses for previously fired requests, worker threads can process them asynchronously, increasing throughput.

However, I am unable to see how Non blocking IO directly benefits low latency application.

I guess "Asynchronous behavior is a great way to avoid contention." If that is the case, low contention means low latency. Hence NIO may help in low latency. Does it make sense?

de.coding.myth
  • 125
  • 1
  • 9
  • Your post consists mostly of *non sequiturs.* It is difficult to find an answerable question here. – user207421 Jan 20 '14 at 22:50
  • @EJP, if it is unclear, I apologize. I simply want to know whether using NIO can benefit a low latency application. If yes, how? – de.coding.myth Jan 21 '14 at 05:53
  • I'd be surprised. With blocking I/O and threads, you have whatever scheduling mechanism the operating system implements, and you can bet that it's very highly tuned. With NIO you provide you own scheduling, in the form of a linear scan over the ready selection keys, and you also get to waste time between selects if you're not very careful. – user207421 Jan 22 '14 at 00:25
  • 1
    Low latency and high troughput are actually conflicting requirements in most cases. You can either optimitze bandwidth usage (e.g. socket connection), which will in turn worsen latency (since all data pending needs to be transferred before a particulare piece of data comes through) or latency (in which case you want to limit the amount of data sent to ensure it goes through quickly). Pick your poison. – Durandal Feb 11 '14 at 19:56

2 Answers2

2

"Asynchronous behavior is a great way to avoid contention." - only when single thread is used. If many threads, contention is unavoidable. You have to use multitrhreading (with or without NIO) to get high throughput and/or low latency.

NIO only helps to keep number of threads low (around the number of available processors) and thus saves memory (each thread consume a lot of memory) and allows enormous number of simultaneous connections, but usually has worse performance than blocking IO.

Alexei Kaigorodov
  • 13,189
  • 1
  • 21
  • 38
1

It all depends on... :)

  • If you have a number of simultaneous threads <= number of CPU cores, then low latency you get can using blocking IO/NIO. Usually, throughput is slightly worse in case of blocking IO/NIO. See for instance: http://vanillajava.blogspot.ru/2011/08/comparing-java-7-async-nio-with-nio.html

  • If you have a large number of thread, thread-per-connection model becomes less preferred over async NIO. Mostly because you pay the cost of the context switching, which may be not so small (time to call the system scheduler, more cache misses, etc). See, for instance: http://www.cs.rochester.edu/u/cli/research/switch.pdf And more threads = more total price :) Here we need async NIO for both latency and throughput.

AnatolyG
  • 1,557
  • 8
  • 14