How to get ThreadPoolExecutor to execute immediately in Java

Question

I have a ThreadPoolExecutor and a Runnable called this.runnable. I run it like this:

ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(1);
executor.execute(this.runnable);

Usually, this.runnable will be called almost immediately (100-200us after). But sometimes if a CPU is busy it can take a while to run.

Is there a way I can "force" this to execute immediately, or suggest to the JVM/ThreadPoolExecutor to prioritse this thread's execution? I need to keep it as a separate thread but at the same time, I need it to start running with urgency.

Deciding when a thread will run is the responsibility of the operating system and cannot be controlled by your java code. In the code you posted, you are only launching one single thread. Did you try simply `new Thread().start()` ? — Abra, Oct 09 '19 at 14:53
@Abra I tried that and it's slower than using a ThreadPoolExecutor. I want to know any hack or method to "suggest" to the OS to prioritise/force this.runnable to start running ASAP. — ABC, Oct 09 '19 at 14:56
Re, "But sometimes if a CPU is busy..." There's not much you can do about that if your application has to run on the same host that is running other, unrelated applications. If you need to control when things happen with sub-millisecond precision, then you probably need to exclude other applications from the platform, and you probably need to run under a Real-Time OS (a.k.a., "RTOS"). — Solomon Slow, Oct 09 '19 at 15:05
_Usually, `this.runnable` will be called almost immediately_ How do you obtain the elapsed time? What exactly do you measure? — Abra, Oct 09 '19 at 15:18
@Abra I log the epoch time (using `Instant.now()`) before I send the thread to the `ThreadPoolExecutor`, and then I log the epoch time again at the beginning of the `run` method. — ABC, Oct 09 '19 at 22:23
@ABC did you also count the time to create the PoolExecutor? — aran, Oct 10 '19 at 15:06
@aran: that should be a one-time hit on startup, so no reason to count it. however you have a point that it is unclear from the code snippet. considering the OP complains about time starting a single thread, that makes me think the executor must be constructed only once. but it would be good to get clarification. — Nathan Hughes, Oct 10 '19 at 15:15
@NathanHughes I assumed it looking at the pool's size. Anyway, even without that time counting as elapsed, I'm really in doubt wether calling execute on a pool is faster than an old `start()`. — aran, Oct 10 '19 at 15:19
@aran: one of the benefits of having a pool is that the threads are already started and ready to go. — Nathan Hughes, Oct 10 '19 at 15:20
@NathanHughes Yes, but creating a thread by yourself will be faster than a pool creation time, so the elapsed time to create the thread by yourself shouldn't count either. Could be interesting creating a benchmark of a `pool` vs manual threads, both groups set as `Thread.MAX_PRIORITY`; — aran, Oct 10 '19 at 15:23

Kaan · Answer 1 · 2019-10-09T18:50:57.267

Is there a way I can "force" this to execute immediately, or suggest to the JVM/ThreadPoolExecutor to prioritse this thread's execution?

No. Also, once a thread starts, there's no way for you to control how it continues to run. You are allowed to set a thread's individual priority, but that's only useful when the JVM is deciding whether to run your thread or some other, possibly lower priority thread – it doesn't allow you to make your thread run whenever or however you want.

Specifically for ThreadPoolExecutor, when you call execute() on an instance of ThreadPoolExecutor the Javadoc says: "Executes the given task sometime in the future." There is no mention of how to influence or control when a thread will start.

I need to keep it as a separate thread but at the same time, I need it to start running with urgency.

Perhaps you could create a thread pool with one (or more) threads ready to go, and use something from Java's built-in concurrency classes (such as Semaphore) as a way to start execution.

UPDATE: Here's an example showing how you could use a Semaphore to "start" a thread.

Semaphore semaphore = new Semaphore(0);
new SomeThread(semaphore).start();

// do other things for a while

// when you're ready for the thread to actually start, call "release()" 
semaphore.release();

Here's the thread itself – it takes a Semaphore as input, then waits until it can successfully acquire it before going to whatever code is next:

class SomeThread extends Thread {
    private Semaphore semaphore;

    SomeThread(Semaphore semaphore) {
        this.semaphore = semaphore;
    }

    @Override
    public void run() {
        try {
            semaphore.acquire();
            // TODO: do the actual thread work here
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

Will using a Semaphore speed up when the thread starts to run, which is the core problem that I need to solve? — ABC, Oct 10 '19 at 03:49
It might, but depends on what your code/app is doing. It seems worth trying it out and comparing with the performance of your existing code. — Kaan, Oct 10 '19 at 16:14

Nathan Hughes · Accepted Answer · 2019-10-10T20:22:43.050

You could use a PriorityBlockingQueue to queue the tasks. That way a high priority task can be given the highest spot in the queue so it runs as soon as a worker is freed up. But I don't like this option because it adds complication and overhead, and low-priority tasks may get starved.

Dinking around with thread priorities usually doesn't work out well. Different platforms implement different priority levels so code relying on priorities may have different behavior on different platforms. Also having different priority threads makes it easy to cause a priority inversion bug. Here is what the book Java Concurrency in Practice (10.3.1) has to say about priorities:

It is generally wise to resist the temptation to tweak thread priorities. As soon as you start modifying priorities, the behavior of your application becomes platform-specific and you introduce the risk of starvation. You can often spot a program that is trying to recover from priority tweaking or other responsiveness problems by the presence of Thread.sleep or Thread.yield calls in odd places, in an attempt to give more time to lower-priority threads.

As others have said before, you can't prevent other processes from using the CPU (you would have to move your application to a platform where it isn't competing with other processes for CPU), and you can't force the OS to schedule and run a particular thread.

You don't have to limit yourself to just one threadpool, there are sometimes good reasons to have multiple pools. JCIP (8.1) advises:

Thread pools work best when tasks are homogeneous and independent. Mixing long-running and short-running tasks risks "clogging" the pool unless it is very large; submitting tasks that depend on other tasks risks deadlock unless the pool is unbounded.

If you have a short task that needs doing right away, and it tends to get stuck getting queued up behind multiple bigger tasks, then you need another dedicated threadpool for high-priority tasks.

What I would do:

First, make sure the threadpool is initialized only once, on startup, and adjust the size of the threadpool to have more than one thread. With only one thread your task may just be blocked behind another one.
If CPU is an issue, consider changing where your application is deployed so it doesn't have to share CPU with other processes, or otherwise fix the hosting so there's more CPU to go around.
If the pool continues to be a bottleneck then make a separate thread pool to use as a fast lane for high priority tasks.

score 0 · Answer 3 · answered Oct 09 '19 at 14:33

You could try to increase the priority of the created thread:

ExecutorService executorService = Executors.newFixedThreadPool(1, runnable -> {
    Thread thread = new Thread(runnable);
    thread.setPriority(Thread.MAX_PRIORITY);
    return thread;
});

How to get ThreadPoolExecutor to execute immediately in Java

3 Answers3