How does newCachedThreadPool reuse threads?

Question

I want to ask for a little more detail to the same question posted by Zeller over a year ago...

The javadoc says that the service returned by Executors.newCachedThreadPool reuses threads. How is this possible?

I get how the queue structure is setup internally, what I don't see is how it reuses threads in the queue.

All examples I've seen have the developer create an instance of their thread and pass it in through the "execute" method.

For example...

ExecutorService executor = Executors.newCachedThreadPool();
for (int i = 0; i < 10; i++) {            
  Runnable worker = new WorkerThread(i);  //will create 10 instances          
  executor.execute(worker);          
} 

I understand that a thread pool can easily manage the life cycle of each thread, but again, I see no methods nor the ability to access or restart any of the threads in the pool.

In the above example, I would then expect that each thread would be started, run, terminated and disposed of by the thread pool, but never reused.

A messaging system would be an example of where you'd need this. Say you have an onMessage handler and you'd like to reuse one of the threads in the pool to handle it, so I'd expect methods like...

worker = executor.getIdleThread;
worker.setData(message);
executor.resubmit(worker);          

or maybe have the ExecutorService acting as a factory class and have it return an instance of your threads, where internally it decides to create a new one or reuse an old one.

ExecutorService executor = Executors.newCachedThreadPool(WorkerThread);
Runnable worker = executor.getThread;
worker.setData(message);

So I'm missing something. It's probably something simple but I've spent the afternoon reading tutorials and examples and still haven't figured it out. Can someone shed some light on the subject?

Answer 1

I was curious too how this was possible since Threads can't be restarted, so I analyzed the code of ThreadPoolExecutor which is the implementation of all the ThreadPool ExecutorService you get through the static constructor.

First of all as stated in the other answer you don't use Threads, but Runnables in ThreadPools, because that would defeat the purpose. So here is a detailed explaination how an ExecutorService reuses Threads:

You usually add a Runnable through submit() which internally calls the execute() method. Basically this adds the runnable to a queue and adds a Worker if none is working ATM

public void execute(Runnable command) {
        ...
        int c = ctl.get();
        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }
        if (isRunning(c) && workQueue.offer(command)) {
            int recheck = ctl.get();
            if (! isRunning(recheck) && remove(command))
                reject(command);
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        else if (!addWorker(command, false))
            reje
ct(command);
    }

The executor maintains a bunch of Worker (inner Class of ThreadPoolExecutor). It has your submitted runnable and a Thread that will be created through the ThreadFactory you maybe set or else just a default one; also the Worker itself is a Runnable, which is used to create the Thread from the factory

private final class Worker
        extends AbstractQueuedSynchronizer
        implements Runnable
    {
...
   Worker(Runnable firstTask) {
     this.firstTask = firstTask;
     this.thread = getThreadFactory().newThread(this);
   }

   public void run() {
      runWorker(this);
   }
...
}

When adding a Worker it gets kickstarted

private boolean addWorker(Runnable firstTask, boolean core) {
    ...
    Worker w = new Worker(firstTask);
    Thread t = w.thread;
    ...
    t.start();
    ...
    return true;
}

the runWorker() method runs in a loop and get with getTask() the runnables you submitted which are queued in the workingQueue and will wait at getTask unitl a timeout happens.

   final void runWorker(Worker w) {
        Runnable task = w.firstTask;
        w.firstTask = null;
        boolean completedAbruptly = true;
        try {
            while (task != null || (task = getTask()) != null) {
                w.lock();
                clearInterruptsForTaskRun();
                try {
                    beforeExecute(w.thread, task);
                    Throwable thrown = null;
                    try {
                        task.run();
                    } catch (RuntimeException x) {
                    ...
                    } finally {
                        afterExecute(task, thrown);
                    }
                } finally {
                    task = null;
                    w.completedTasks++;
                    w.unlock();
                }
            }
            completedAbruptly = false;
        } finally {
            processWorkerExit(w, completedAbruptly);
        }
    }

Here is the getTask() method

private Runnable getTask() {
   ...
        try {
            Runnable r = timed ?
                workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
                workQueue.take();
            ..
        } catch (InterruptedException retry) {
            ...
        }
    }
}

---

tl;dr So basically the Threadpool maintains worker threads that run in loops and execute runnables given by a blocking queue. The worker will be created and destroyed due to demand (no more tasks, a worker will end; if no free worker and < maxPoolSize then create new worker). Also I wouldn't call it "reuse" more the thread will be used as a looper to execute all runnables.

Answer 2

I understand that a thread pool can easily manage the life cycle of each thread, but again, I see no methods nor the ability to access or restart any of the threads in the pool.

The management of threads is done internally. The ExecutorService interface only provides the externally visible methods.

The javadoc of newCachedThreadPool simply states

Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. [...] Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources. [...]

So that is the guarantee you get. If you want to know how it is implemented, you can look at the source code, in particular, the code of ThreadPoolExecutor. Basically, idle threads will terminate after not executing a task after some time.