Thread-Pool with multiple limits

Question

I want a Thread-Pool that provides a maximum of X Threads to process tasks, so far no problem. However each submitted Task can specify an IO-Target which is specifically limited (say Y).

So a submitted IOTask returns the target "google.com" with limit 4 (Y) and the pool has a global limit 16 (X). I want to submit 10 google.com-tasks where only 4 are processed in parallel and the pool has 12 threads free for other tasks.

How can I achieve this?

Answer 1

Implementing this functionality is not simple since you will either need to have separate queues per target (so the waiting code becomes far more complicated), or one queue from which you then skip over targets that are at capacity (incurring a performance overhead). You can try to extend ExecutorService to achieve this, but the extension appears to be non-trivial.

Updated answer / solution:

After thinking about this a little bit more, the easiest solution to the blocking problem is to have both a blocking queue (as per normal) as well as a map of queues (one queue per target, as well as a count of available threads per target). The map of queues is used only for tasks that have been passed over for execution (due to too many threads already running for that target) after the task is fetched from the regular blocking queue.

So the execution flow would look like this:

1. task is submitted (with specific target) by calling code.
2. Task is put onto the blocking queue (likely wrapped here by your own task class that includes target information).
3. thread (from the thread pool) is waiting on the blocking queue (via take()).
4. thread takes the submitted task.
5. thread synchronizes on lock.
6. thread checks the available count for that target.

7. if the available count > 0

   • then the thread decreases count by 1, releases lock, runs task.
   • else the thread puts the task into the map of target to task queue (this map is the passed over task map), releases lock, and goes back to waiting on the blocking queue.

8. when a thread finishes execution of a task it:

   • synchronizes on lock.
   • checks the count for the target it just executed.
   • if the count == 0
     • then check the passed over task map for any tasks for this target, if one exists, then release lock and run it.
   • if count was not 0 or no task for the same target was on the passed over map / queue, then increase the available count (for that target), release the lock, and go back to waiting on the blocking queue.

This solution avoids any significant performance overhead or having a separate thread just to manage the queue.

Answer 2

You can wrap two ExecutorService instances in a custom class and manually manage the submission of tasks as follows:

class ExecutorWrapper {

    private ExecutorService ioExec = Executors.newFixedThreadPool(4);
    private ExecutorService genExec = Executors.newFixedThreadPool(12);

    public Future<?> submit(final IOTask task) {
        return ioExec.submit(task);
    }

    public Future<?> submit(final Runnable task) {
        return genExec.submit(task);
    }
}

interface IOTask extends Runnable {}

This allows you to use 4 threads to do the IO operations and leaves the other 12 threads to service the other tasks.

Answer 3

Thinking through some of the answers in a more specific way.

1. You'll need your own BlockingQueue that can separate the different types of tasks and return the desired Runnable depending on an internal counter.

2. Extend ThreadPoolExecutor and implement beforeExecute and afterExecute.

When beforeExecute is called it would increment a counter within the queue if the Runnable was of type X. When afterExecute was called it would decrement that counter.

Within your queue you would then return the appropriate Runnable depending on the value of the counter, I believe the take method is where you would do this.

There are some synchronization issues here that have to be thought through completely to ensure that the counter never gets over 4. Unfortunately once you are within beforeExecute it's too late but being able to simply know how many of what task are running at a given time might get you started.

Answer 4

Hmmm... I'm afraid the exiting ExecutorService does not allow such fine-grain control. You'll probably need to either extend the ExecutorService class to add this functionality yourself or use two separate fixed thread pools, one with a capacity of 4, the other with a capacity of 12.

Answer 5

An idea for this could be to extend the ExecutorService and in your class have two ThreadPools, one with a capacity of 4 and the other with a capacity of 12.

Then implement the methods that you need and based on the IOTasks submitted you can direct the tasks to which pool you would like it to go to.

Answer 6

Use a counter for total threads and HashMap which counts the number of threads currently attempting to access site X. When you want to start a new thread, call a synchronized method which checks waits (wait() inside a while loop) until the the number of threads in the hash map is less than 4 and the total number of threads is less than 16. Then increment both counters and start the thread. When the thread finishes, it should call a second synchronized method which decrements the counters and calls notify()