create new Thread for every file sent by client (sockets)

Question

I am working on application with sockets. Client is watching specific directory and when user adds a file in there, the client is sending this file to the specific directory on the server side through sockets. Now i have implemented semaphore to have many files being sent concurrently to the server. And here is where my problem begins. I am using while loop and when client is trying to send some file, server is creating all threads it can. In such case 4, because of this :

static Semaphore semaphore = new Semaphore(4); 

while (true) {       
    MyRunnable t1 = new MyRunnable("Name" + number++);

    Thread thread = new Thread(t1);
    thread.start();                     
}

This is my Thread class and what happens inside

public class MyRunnable implements Runnable {

        String name;

        public MyRunnable(String name) {
            this.name = name;
        }

        private void saveFile(String path) throws IOException {

            byte[] buffer = new byte[4096]; //4096 16384

            String fileName = dis.readUTF();
            int fileSize = (int) dis.readLong();

            int read = 0;
            int totalRead = 0;

            FileOutputStream fos = new FileOutputStream(path + fileName);
            System.out.println("Name of the file " + fileName);

            while ((read = dis.read(buffer, 0, Math.min(buffer.length, fileSize))) > 0) {
                totalRead += read;
                fileSize -= read;
                System.out.println("Read" + totalRead + " bytes.");
                fos.write(buffer, 0, read);
            }

        }

        public void run() {
            try {
                semaphore.acquire();
                System.out.println(name + " : got the permit!");
                System.out.println("available Semaphore permits : "
                        + semaphore.availablePermits());

                try {
                    saveFile(pathToFiles + login + "\\");
                } finally {

                    // calling release() after a successful acquire()
                    System.out.println(name + " : releasing lock...");
                    semaphore.release();
                    System.out.println(name + " : available Semaphore permits now: "
                            + semaphore.availablePermits());

                }

            } catch (IOException | InterruptedException e) {
                e.printStackTrace();
            }
        }
    }

This is my output when nothing even is send yet :

Name0 : got the permit!
Name1 : got the permit!
Name3 : got the permit!
Name2 : got the permit!
available Semaphore permits : 0
available Semaphore permits : 0
available Semaphore permits : 0
available Semaphore permits : 0

Here is fragment of my client code

public void watchDirectory(Path path) throws IOException, InterruptedException {
        WatchService watchService
                = FileSystems.getDefault().newWatchService();

        path.register(
                watchService,
                StandardWatchEventKinds.ENTRY_CREATE
                //     StandardWatchEventKinds.ENTRY_DELETE,
               // StandardWatchEventKinds.ENTRY_MODIFY
                );

        WatchKey key;
        while ((key = watchService.take()) != null) {
            for (WatchEvent<?> event : key.pollEvents()) {
                System.out.println(
                        "Event kind:" + event.kind()
                                + ". File affected: " + event.context());
                Runnable runnable = () -> {
                    System.out.println("Inside : " + Thread.currentThread().getName());
                    File file = new File(pathToFiles + login + "\\" + event.context());
                    try {
                        Thread.sleep(50);
                        sendFile(file);
                    } catch (IOException | InterruptedException e) {
                        e.printStackTrace();
                    }

                };
                
                System.out.println("Creating Thread...");
                Thread thread = new Thread(runnable);

                System.out.println("Starting Thread...");
                thread.start();
            }
            key.reset();
        }
    }

    public void sendFile(File file) throws IOException {

        FileInputStream fis = new FileInputStream(file);
        byte[] buffer = new byte[4096];  //4096 16384

        // writing name
        dos.writeUTF(file.getName());
        // writing length
        dos.writeLong(file.length());

        System.out.println(file.getName() + " " + file.length());

        int count;

        while ((count = fis.read(buffer)) > 0) {
            dos.write(buffer, 0, count);
        }

    }

I want to have one thread working on one file client sends. And when client sends more files i want threads to work concurently. Is there a good way to do it ? Would be grateful for any help.

Answer 1

If you want to re-use a set of 4 threads, the best way is to use a fixed thread-pool executor:

ExecutorService threadPoolExecutor = Executors.newFixedThreadPool(4);

threadPoolExecutor.submit( new SomeRunnable() );

(You can read more about this method in the documentation.)

In your case, you would create the ExecutorService outside your directory watcher, and then submit() a task to it inside your watcher code at the spot where it wakes up to do something.

For a good example of how to watch a directory in a thread of its own, see this question. Following that example, your directory watcher class would look something like this:

public class DirWatcher implements Runnable {

    private final Path dir;
    private final WatchService watcher;
    private final WatchKey key;

    // thread pool for handling files in the watched directory
    ExecutorService threadPoolExecutor = Executors.newFixedThreadPool(4);

    @SuppressWarnings("unchecked")
    static <T> WatchEvent<T> cast(WatchEvent<?> event) {
        return (WatchEvent<T>) event;
    }

    public DirWatcher(Path dir) throws IOException {
        this.dir = dir;
        this.watcher = FileSystems.getDefault().newWatchService();
        this.key = dir.register(watcher, ENTRY_CREATE);
    }

    public void run() {
        try {
            for (;;) {
                WatchKey key = watcher.take();

                if (this.key != key) {
                    continue;
                }

                for (WatchEvent<?> event : key.pollEvents()) {
                    WatchEvent<Path> ev = cast(event);
                    String fileName = dir.resolve( ev.context() ).toString();

                    // handle event
                    threadPoolExecutor.submit( new SomeTask( fileName ) );
                }

                // reset key
                if (!key.reset()) {
                    break;
                }
            }
        } catch (InterruptedException x) {
            return;
        }
    }
}

You can see the ExecutorService instance and the submit() call where your directory watcher responds to a new file being created, with a placeholder "SomeTask" for your Runnable which you'll call for whatever work you want to do on the file.

You'd call it like so:

public static void watchThatDirectory(String dirName) throws IOException, InterruptedException {

    Path dir = Paths.get(dirName);
    DirWatcher watcher = new DirWatcher(dir);

    ExecutorService executor = Executors.newSingleThreadExecutor();
    Future<?> future = executor.submit(watcher);
    executor.shutdown();
}

And now your directory watcher thread is running in the background, and will launch new threads (to a maximum of 4) to handle the appearance of new files in the watched directory. Meanwhile, you can do other work in your main thread.