How to avoid concurrent execution of a time-consuming task without blocking?

Question

I want to efficiently avoid concurrent execution of a time-consuming task in a heavily multi-threaded environment without making threads wait for a lock when another thread is already running the task. Instead, in that scenario, I want them to gracefully fail (i.e. skip its attempt to execute the task) as fast as possible. In other words: I need to make any attempt to launch the task again when it is already in progress to back off immediately, preferably without a synchronization cost.

To illustrate the idea considerer this unsafe (has race condition!) code:

private static boolean running = false;

public void launchExpensiveTask() {
    if (running) return; // Do nothing

    running = true;
    try {
        runExpensiveTask();
    } finally {
        running = false;
    }
}

I though about using a variation of Double-Checked Locking (consider that running is a primitive 32-bit field, hence atomic, it could work fine even for Java below 5 without the need of volatile). It could look like this:

private static boolean running = false;
private static Object execLock = new Object();

public void launchExpensiveTask() {
    if (running) return; // Do nothing

    synchronized (execLock) {
        if (running) return;

        running = true;
        try {
            runExpensiveTask();
        } finally {
            running = false;
        }
    }
}

Maybe I should also use a local copy of the field as well (not sure now, please tell me).

But then I realized that anyway I will end with an inner synchronization block, that still could hold a thread with the right timing at monitor entrance until the original executor leaves the critical section (I know the odds usually are minimal but in this case we are thinking in several threads competing for this long-running resource).

So, could you think in a better approach?

EDIT: I previously omitted part of the context, for correctness here I need to maintain a lock during execution to hold other methods trying to change some internal shared state. To be fair I upvoted helpful answers so far including both cases: with and without the need of a lock after starting the task.

Answer 1

I think this makes a little more sense:

 static volatile Boolean running = false;

    public static void launchTask()
    {
        synchronized(running)
        {
            if(running) return;
            running = true;
        }
            //DOSTUFF
            running = false;
    }

Because you really only need to be synchronized on setting the boolean: If several threads ask at the same time, the first one will set running to true and the rest will all return.

However, there may be a better overall pattern for your design. What if threads submitted requests to a queue,(An ExecutorService?) got Future or ListenableFuture (from Guava) objects, then continued to do other stuff until the futures finished their calculations?

Answer 2

With the help of Lock#tryLock() (API available since Java 5), we can do it non-blocking:

private static boolean running = false;
private static Lock execLock = new ReentrantLock();

public void launchExpensiveTask() {
    if (running) return; // fast exit without sync

    if (!execLock.tryLock()) return; // quit if lock is not free

    try {
        running = true;
        runExpensiveTask();
    } finally {
        running = false;
        execLock.unlock();
    }

}

In case that you do not need to hold a lock during task execution, take a look at the following code:

private static boolean running = false;
private static Object execLock = new Object();

private boolean start() {
    synchronized (execLock) {
        boolean ret = running;
        running = true;
        return ret;
    }
}

private void end() {
    synchronized (execLock) {
        running = false;
    }
}

public void launchExpensiveTask() {
    if (running) return; // fast exit without sync

    if (start()) return; // already running, do nothing

    try {
        runExpensiveTask();
    } finally {
        end();
    }
}

Answer 3

Disregard my other answer. But what you are looking for is this. http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/Semaphore.html

By using semaphores. The simplest way to think of a semaphore is to consider it an abstraction that allows n units to be acquired, and offers acquire and release mechanisms. TryAcquire is key because according to the documentation of java - Acquires a permit from this semaphore, only if one is available at the time of invocation.Try it for yourself.

    private Semaphore semaphore = new Semaphore(1);

    public void launchExpensiveTask() {
        if (semaphore.tryAcquire()) {
            try {
               runExpensiveTask();
            } finally {
               semaphore.release();
            }
        }

    }

Answer 4

Keep in mind that for most people the following solution will be preferable:

private static Lock execLock = new ReentrantLock();

public void launchExpensiveTask() {
    if (!execLock.tryLock()) return; // skip if already running

    try {
        runExpensiveTask();
    } finally {
        lock.unlock();
    }
}

Please note that particular cases like the one exposed in this question are very rare, usually the performance of synchronization primitives is more than enough and the simpler the code the better. Avoid premature optimization, use this unless you are certain that it is not working for you.

Answer 5

Update: after discussing with the question owner, here is the final proposed solution, with code:

package toys;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;

public class TwoQueues {

//tweak it for your purpose.
private final int CPU_COUNT = 4;
private BlockingQueue<Runnable> lightTaskQueue = new LinkedBlockingDeque<Runnable>();
private ThreadPoolExecutor lightExecutor = new ThreadPoolExecutor(CPU_COUNT, CPU_COUNT, 60L, TimeUnit.SECONDS, lightTaskQueue);
private BlockingQueue<Runnable> heavyTaskQueue = new LinkedBlockingDeque<Runnable>();
private ThreadPoolExecutor heavyExecutor = new ThreadPoolExecutor(1, 1, 60L, TimeUnit.SECONDS, heavyTaskQueue);

public static class SampleLightTask implements Runnable {

    @Override
    public void run() {
        System.out.println("I am " + this + " and running fast!");
    }

}

private static AtomicBoolean heavyTaskRunning = new AtomicBoolean();

public static class SampleHeavyTask implements Runnable {

    @Override
    public void run() {
        try {
            heavyTaskRunning.set(true);
            System.out.println("I am " + this + " and running quite slow!");
            final long start = System.currentTimeMillis();
            while (true) {
                //burn the CPU for ten senconds.
                if (System.currentTimeMillis()-start >= 10000L)
                    break;
            }
        } finally {
            heavyTaskRunning.set(false);;
        }
    }

}

public void shutDownNow() {
    this.lightExecutor.shutdownNow();
    this.heavyExecutor.shutdownNow();
}

public void runOrQueueLightTask(SampleLightTask lightOne) {
    this.lightExecutor.execute(lightOne);
}

public void runOrQueueHeavyTask(SampleHeavyTask heavyOne) {
    if (heavyTaskRunning.get()) {
        System.out.println("running, skipped new one: " + heavyOne);
        return;
    }

    this.heavyExecutor.execute(heavyOne);
}

public static void main(String[] args) throws Exception {
    TwoQueues q = new TwoQueues();

    final long start = System.currentTimeMillis();

    //Run the queues for 30 seconds, add CPU-light and CPU-weight tasks
    //every second.
    while (System.currentTimeMillis()-start<=30*1000L) {
        q.runOrQueueHeavyTask(new SampleHeavyTask());
        q.runOrQueueLightTask(new SampleLightTask());
        Thread.sleep(1000L);
    }

    q.shutDownNow();
}
}

And the running output:

I am toys.TwoQueues$SampleHeavyTask@6d0cecb2 and running quite slow!
I am toys.TwoQueues$SampleLightTask@6b87d20c and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@2ce07e6b
I am toys.TwoQueues$SampleLightTask@7fa0d111 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@16fdf48d
I am toys.TwoQueues$SampleLightTask@5fbd7d0e and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@115d533d
I am toys.TwoQueues$SampleLightTask@59c27402 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@6d4e5d57
I am toys.TwoQueues$SampleLightTask@33d232d1 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@79ec3264
I am toys.TwoQueues$SampleLightTask@1e081c5 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@3a67ad79
I am toys.TwoQueues$SampleLightTask@6cae00e3 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@13bc6ed3
I am toys.TwoQueues$SampleLightTask@380fe8c4 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@1c7ab89d
I am toys.TwoQueues$SampleLightTask@3cee5a06 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@44585f2a
I am toys.TwoQueues$SampleLightTask@5cfe174 and running fast!
I am toys.TwoQueues$SampleLightTask@12da89a7 and running fast!
I am toys.TwoQueues$SampleHeavyTask@49833c9c and running quite slow!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@47004b78
I am toys.TwoQueues$SampleLightTask@645ad7b2 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@8071a97
I am toys.TwoQueues$SampleLightTask@a62b39f and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@55fe910c
I am toys.TwoQueues$SampleLightTask@3be4d6ef and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@2cdb03a1
I am toys.TwoQueues$SampleLightTask@5ecb5608 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@777d57d6
I am toys.TwoQueues$SampleLightTask@4611dfe3 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@3f81d405
I am toys.TwoQueues$SampleLightTask@6486b4d5 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@47ca3f82
I am toys.TwoQueues$SampleLightTask@2f0f94a0 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@27e6ac83
I am toys.TwoQueues$SampleLightTask@1947e0ec and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@3dffb2eb
I am toys.TwoQueues$SampleLightTask@5e3b8219 and running fast!
I am toys.TwoQueues$SampleLightTask@14da67a4 and running fast!
I am toys.TwoQueues$SampleHeavyTask@eca4aae and running quite slow!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@2eced18
I am toys.TwoQueues$SampleLightTask@10c1c428 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@213526b0
I am toys.TwoQueues$SampleLightTask@287efdd8 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@294b84ad
I am toys.TwoQueues$SampleLightTask@1cf38f09 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@3a33a6b8
I am toys.TwoQueues$SampleLightTask@150697e2 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@63dd8136
I am toys.TwoQueues$SampleLightTask@634e3372 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@2313b44d
I am toys.TwoQueues$SampleLightTask@62a23d38 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@9615a1f
I am toys.TwoQueues$SampleLightTask@5663ae08 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@2a36bb87
I am toys.TwoQueues$SampleLightTask@6f51b1b7 and running fast!
running, skipped new one: toys.TwoQueues$SampleHeavyTask@5c6a9e79
I am toys.TwoQueues$SampleLightTask@5bca4955 and running fast!

////////////////////////////////// Old Answer ////////////////////////////////////

If I understand your requirement correctly, there are two types of tasks: type A, CPU intensive, executed in a serials manner, and possibly modify some global states that are not thread safe; type B, not CPU intensive, and need to completes them as fast as possible.

Why not using two thread pools and two queues for this? this is a perfect match. For tasks of type A, schedule them in a thread pool with max concurrency set to 1; and for the type B, in another thread pool with max concurrency set to your CPU core/thread numbers or whatever suitable for your need. There is even no need for "check & gracefully fail" here.

I used to write a lot of these primitive, low-level concurrent, threading stuff myself, but the time the thread pool becomes standard library in JDK, I never go back again, both on server side(EE) and client side(Android here). The design is clean, the performance is good, much less code, and of course, much less bugs. Debugging a concurrency related bug is never easy.