Java associate object with each thread in Executor

Question

I have a bunch of runnables that I want to execute via a thread pool. However, each runnable also writes some result to some file. So right now, the runnable's interface is simple:

class MyRunnable implements Runnable {
    ...
    MyRunnable(BufferedWriter writer, Task t) {
       this.writer = writer;
       this.task = t;
    }

    public void run() {
       ...
       this.writer.write(SOME_DATA);
    }
}

However, what I want is to associate one BufferedWriter (in other words, one output file) with each of the thread in the Executor pool. However, I'm calling the .execute function using ExecutorService like the following:

BufferedWriter[] writers = initialize to 20 BufferedWriters;
ExecutorService executor = Executors.newFixedThreadPool(20);
for (Task t : tasks) {
    MyRunnable runnable = new MyRunnable(WHAT SHOULD GO HERE?, t)
    executor.execute(runnable);
}

I don't know which thread the executor will assign to run a given task, so I don't know which BufferedWriter I should provide to the runnable. How can I ensure that each thread managed by the ExecutorService is associated with one object (in this case a BufferedWriter)?

score 3 · Answer 1 · answered Oct 20 '13 at 23:50

There is a class called ThreadLocal for this.

e.g.

ThreadLocal<Type> t = new ThreadLocal<>() {
    @Override protected Type initialValue() {
        return new Type(Thread.currentThread.getName());
}

This will lazy-initialize a Type each time a new thread tries to access t.

The last time I used this class was simply in a class to figure out how many threads a certain machine runs best at. (The answer is usually "the number of cores," but I wanted to make sure that virtual cores drove this number and not physical cores). So I wrote a task in which every thread just spammed an AtomicInteger counter. But all the threads were fighting over one counter which created a lot of overhead that went into dealing with thread-contention, so I created a threadlocal counter, so the threads could spam their own counters without interference from other threads.

The use cases for it are somewhat obscure since most good multithreaded designs avoid this, but there are times for it of course.

so nice, just what I need. Thanks for pointing this class out :) — Jin, Oct 20 '13 at 23:55

score 2 · Accepted Answer · edited May 23 '17 at 12:28

... I want is to associate one BufferedWriter (in other words, one output file) with each of the thread in the Executor pool...

@djechlin's answer about ThreadLocal is good but the problem with it is that you can't get access to the BufferedWriter to close() them when the threads finish running the last task.

An alternative answer can be seen here:

Threadpool with persistent worker instances

In it I recommend creating your own BlockingQueue of tasks and forking one task per thread and having those threads get the tasks from your queue. So the thread run method would be something like:

private final BlockingQueue<MyRunnable> queue = new ArrayBlockingQueue<>();
// if you want to shutdown your threads with a boolean
private volatile boolean shutdown;
...

// threads running in the `ExecutorService` will be doing this run() method
public void run() {
    // this allows them to maintain state, in this case your writer
    BufferedWriter writer = ...;
    while (!shutdown && !Thread.currentThread.isInterrupted()) {
        // they get their tasks from your own queue
        MyRunnable runnable = queue.take();
        // if you are using a poison pill but you'll have to add X of them
        if (runnable == STOP_OBJECT) {
            break;
        }
        runnable.run();
    }
    writer.close();
}

Telling the threads when they are done is a little tricky here. You could add a "poison pill" object to the queue but you would have to add the same number of objects to the queue as there are threads running.

lol, while implementing @djechlin's answer I came across the exact problem of not being able to close the writers -.- So what you recommend is essentially manage my own pool of threads, and each thread pulls from a synchronized source? — Jin, Oct 21 '13 at 00:11
Well you can still use an `ExecutorService` to manage the threads which is always recommended @Jin. But have they pull their tasks from your `BlockingQueue`. — Gray, Oct 21 '13 at 00:14
Lol I just thought of another easier but hacky solution.. run "close buffered writer" runnable after everything completes xD — Jin, Oct 21 '13 at 00:21
THe problem is that you won't be guaranteed to hit every thread @Jin. It could be that the same thread runs all of them depending on race conditions. There is no guarantee of FIFO thread access. — Gray, Oct 21 '13 at 00:22

Java associate object with each thread in Executor

2 Answers2

Linked