Java Server having many clients connect without bottlenecking

Question

So what I'm trying to do is have a socket that receives input from the client, put the client into the queue and then return a message to each client in the queue when my algorithm returns true.

This queue should support a few hundred clients at once but at the same time not bottle neck the server so it can actually do what its supposed to do.

This is what i have so far:

private static final int PORT = 25566;
private static final int THREADS = 4;
private ExecutorService service;

public void init() throws IOException, IllegalStateException {
    ServerSocket serverSocket;
    serverSocket = new ServerSocket(PORT);
    service = Executors.newCachedThreadPool();
    Socket socket;
    while(true) {
        socket = serverSocket.accept();
        System.out.println
            ("Connection established with " + socket.getInetAddress().toString());
        service.execute(() -> {
            Scanner scanner = null;
            PrintWriter output = null;
            String line = null;
            try {
                scanner = new Scanner(new InputStreamReader(socket.getInputStream()));
                output = new PrintWriter(socket.getOutputStream());
            } catch(IOException e) {
                e.printStackTrace();
            }
            try {
                if (scanner == null || output == null)
                    throw new IllegalStateException("Scanner/PrintWriter is " + "null!");
                line = scanner.nextLine();
                while (line.compareTo("QUIT") != 0) {
                    /* This is where input comes in, queue for the algorithm, 
                       algorithm happens then returns appropriate values */

                    output.flush();
                    line = scanner.nextLine();
                }
            } finally {
                try {
                    System.out.println
                        ("Closing connection with " + socket.getInetAddress().toString());
                    if(scanner != null) {
                        scanner.close();
                    }
                    if(output != null) {
                        output.close();
                    }
                    socket.close();
                } catch(IOException e) {
                    e.printStackTrace();
                }
            }
        });
    }
}

Now what I think will happen with this, is if the queues do reach high enough levels, my thread pool will completely bottleneck the server as all of the threads are being put to use on handling the clients in the queue and there won't be enough processing for the algorithm.

EDIT: After a bunch of testing, I think it will work out if in the algorithm it returns the value then disconnects, not waiting for user response but having the users client reconnect after certain conditions are met.

Take a look at [netty](http://netty.io/). It makes writing scalable client/server applications much easier. — Sean Bright, May 02 '17 at 20:51
You need to handle all the I/O to each accepted client in a separate thread per client. See the Custom Networking section of the Java Tutorial. — user207421, May 02 '17 at 21:09
Yeah but if you have >100 clients, one thread per client isnt realistic — Alex Pawelko, May 02 '17 at 22:01
@BobbyMacBob Nonsense. That was conclusively disproved more than ten years ago. There are thread-based servers out there with hundreds of thousands of threads. A few hundred is nothing. — user207421, May 02 '17 at 23:04

score 1 · Accepted Answer · answered May 03 '17 at 02:46

Your bottleneck is unlikely to be processing power unless you are machine limited. What's more likely to happen is that all the threads in your thread pool are consumed and end up waiting on input from the clients. Your design can only handle as many clients at once as there are threads in the pool.

For a few hundred clients, you could consider simply creating a thread for each client. The limiting resource for the number of threads that can be supported is typically memory for the stack that each thread requires, not processing power; for a modern machine with ample memory, a thousand threads is not a problem, based on personal experience. There may be an operating system parameter limiting the number of threads which you may have to adjust.

If you need to handle a very large number of clients, you can set up your code to poll sockets for available input and do the processing only for those sockets that have input to be processed.

Java Server having many clients connect without bottlenecking

1 Answers1