Parallel Processing in Java - schedule tasks to Threads

Question

I'm implementing a Java App which performs parallel processing: When my app starts, the 4 threads that belong to a Thread Pool are created. Then, I start receiving objects that are added to a LinkedBlockingQueue. Each object has an ID, and depending on the ID, a pre-defined task is performed. My "Queue Consumers" are those 4 Threads that were initialized, and they begin to take the objects out of the queue, and perform the tasks that corresponds to each object.

It is possible to define in which Thread each object will be processed? I want to "schedule tasks to each Thread".

For example:

When the first object is taken from the queue, he will be processed by Thread1.
If the second object as a different ID than the first one, it will be processed by Thread2.
If the third object has the same ID than the first object that was taken, it will "go to Thread1".

How can I implement this?

Thanks

I suppose you could, but that sort of defeats the purpose of having one queue and multiple threads. I assume that the point to having multiple threads would be to increase throughput by using parallel processing. So ideally, any idle thread would pick up new work immediately and start processing. In the scenario you are describing, it seems like it would be possible for threads to be idle because the next task in the queue 'belongs' to some other thread that is currently busy. Are you sure you don't want 4 separate queues? — matt forsythe, Nov 14 '13 at 14:46

score 0 · Answer 1 · answered Nov 14 '13 at 14:44

0

Instead of one queue and a thread pool you could have a queue for each individual thread. Naturally, you would have to manage all this yourself or use a software package.

answered Nov 14 '13 at 14:44

edharned

1,884
1
19
20

score 0 · Accepted Answer · answered Nov 14 '13 at 14:52

Let all threads have their own queue. You might implement a new thread like MainThread or DispatcherThread which takes objects from your LinkedBlockingQueue and decides which WorkerThread to process it. Something like :

private void decideQueue(String objectId) { // actually decides which thread to process it.
    Queue queue = getQueueIfObjectIdPreviouslyUsed(objectId);
    if (queue == null) {
        queue = getNextAvailableQueue();
    }

    queue.enqueue();

    //.. 
    //don't forget to dequeue from the main queue
    //..
}

To decide whether an onjectId is previously processed or not you might use a Map that keeps objectId as key and associated Queue as value. And you do the necessary work in getQueueIfObjectIdPreviouslyUsed method.

Parallel Processing in Java - schedule tasks to Threads

2 Answers2