Java Single Client Thread for multiple Server communication for IOT project

Question

I want to create a client Thread for TCP IP communication. But the client thread should be able to communicate with minimum 10000 servers in the same time. As I want to communicate with millions of servers(Practically GPRS based field devices), its not a good practice to start millions of client threads and manage it. I have tried with NIO Selector to do this. But I found that its only possible for the server thread. ThreadPoolExecutor also I have tried. That also failed to manage huge number of connections in the same time. Please give a good suggestion to manage huge number of servers without starting too many threads.

Answer 1

Maybe use something non-blocking like Play framework (uses either Akka or Netty under the hood) or Vert.x to handle those requests. They use dispatch the work to your custom thread pools in order to avoid blocking.

https://www.lightbend.com/blog/why-is-play-framework-so-fast

Because Play is non-blocking, threads are not blocked when the network is slow, but are free to serve another HTTP request–and because Play is stateless, there is no session information tied to the thread that would confuse it.

It is pretty much the same thing for Vert.x

Maybe also have a look at Erlang and Go (Go routine), also look into UDP. In any case the hardware (network, RAM, SSD) will ultimately be the bottleneck. As others here and other answers stated Netty is the go to library, or something using it may be of interest.

Answer 2

If you need to perform network I/O with pure Java, you have three options: old I/O, new I/O (nio) and new I/O 2 (nio2). Old I/O uses blocking calls, so basically you'll need one thread per connection. You don't want to do that, so you should use nio or nio2. There's actually no restriction on selector to be usable with only server, you can use it for client connections.

Also please note that using nio or nio2 is not the easiest task in the world. Consider using netty library which was created exactly for your use-case: to help writing scalable network applications. It's not the easiest library to use as well, but I still recommend it, because it implements a lot of things for you. Also there are a lot of protocol implementations for netty, so chances are, you won't even need to write that much code.

Most of network examples are about servers indeed, because that's what people generally write. But it's perfectly possible to write non-blocking network clients in the same way.