How to access running threads inside ThreadPoolExecutor?

Question

I have a queue of running threads and would like to expose some of its data while it is executed, to monitor the process.

ThreadPoolExecutor provides access to its queue and I can iterate through these objects to call my overridden toString() method, but these are only threads that are waiting for execution.

Is there a way to access threads that are currently running to call my method? Or maybe there's a better approach for this task in general?

To clarify a bit more about the purpose, here's some code of general idea:

public class GetDataTask implements Runnable {
    private String pageNumber;
    private int dataBlocksParsed;
    private String source;
    private String dataType;


    public GetDataTask(String source, String dataType) {
        this.source = source;
        this.dataType = dataType;
    }

    @Override
    public void run() {
        //do stuff that affects pageNumber and dataBlocksParsed
    }

    @Override
    public String toString() {
        return "GetDataTask{" +
            "source=" + source +
            ", dataType=" + dataType +
            ", pageNumber=" + pageNumber +
            ", dataBlocksParsed=" + dataBlocksParsed +
            '}';
    }
}

and a class holding the executor:

public class DataParseManager {
    private static ThreadPoolExecutor executor = new ThreadPoolExecutor(100, 100, 20, TimeUnit.SECONDS, new ArrayBlockingQueue<>(300));

    public void addParseDataTask(String source, String dataType) {
        executor.execute(new GetDataTask(source, dataType));
    }

    // here's the method that I need
    public String getInfo() {
        StringBuilder info = new StringBuilder();
        //and here's the method that I'm missing - executor.getActiveThreads()
        for (Runnable r : executor.getActiveThreads()) {
            info.append(((GetDataTask) r).toString()).append('\n');
        }
        return info.append(executor.toString()).toString();
   }
}

Answer 1

How about wrap Runnable like this.

static class MonitorRunnable implements Runnable {

    static final List<Runnable> activeTasks = Collections.synchronizedList(new ArrayList<>());

    private final Runnable runnable;

    public MonitorRunnable(Runnable runnable) {
        this.runnable = runnable;
    }

    @Override
    public void run() {
        activeTasks.add(runnable);
        runnable.run();
        activeTasks.remove(runnable);
    }
}

and

public class DataParseManager {
    private static ThreadPoolExecutor executor = new ThreadPoolExecutor(100, 100, 20, TimeUnit.SECONDS, new ArrayBlockingQueue<>(300));

    public void addParseDataTask(String source, String dataType) {
        executor.execute(new MonitorRunnable(new GetDataTask(source, dataType)));
    }

    // here's the method that I need
    public String getInfo() {
        StringBuilder info = new StringBuilder();
        //and here's the method that I'm missing - executor.getActiveThreads()
        synchronized (MonitorRunnable.activeTasks) {
            for (Runnable r : MonitorRunnable.activeTasks) {
                info.append(((GetDataTask) r).toString()).append('\n');
            }
        }
        return info.append(executor.toString()).toString();
   }
}

Answer 2

Whenever you add a thread to the queue, also add it to a second data structure, say a HashSet. Then, if you need to access a running thread, you could check the ExecutorService's queue to find the Threads that are still awaiting execution: every thread in your HashSet that is not still in the ExecutorService's queue is currently running.

Answer 3

Like I wrote in comment. I'd do an active update on a shared statistics object approach:

I'd change the Task like this:

public class GetDataTask implements Runnable {
    private String pageNumber;
    private int dataBlocksParsed;
    private String source;
    private String dataType;
    HashMap<GetDataTask,String> statistics


    public GetDataTask(String source, String dataType, HashMap<GetDataTask,String> statistics) {
        this.source = source;
        this.dataType = dataType;
        this.statistics = statistics;
    }

    @Override
    public void run() {
        // you'll probably want to immediately have stats available:
        statistics.put(this, this.toString());

        //do stuff that affects pageNumber and dataBlocksParsed
        // vv this will probably be inside your "do stuff" loop
        statistics.put(this, this.toString());
        // loop end

        // if you do not want stats of finished tasks, remove "this" here.
    }

    @Override
    public String toString() {
        return "GetDataTask{" +
            "source=" + source +
            ", dataType=" + dataType +
            ", pageNumber=" + pageNumber +
            ", dataBlocksParsed=" + dataBlocksParsed +
            '}';
    }
}

and the manager:

public class DataParseManager {
    private static ThreadPoolExecutor executor = new ThreadPoolExecutor(100, 100, 20, TimeUnit.SECONDS, new ArrayBlockingQueue<>(300));

    private HashMap<GetDataTask,String> stats = new ConcurrentHashMap<GetDataTask,String>();       

    public void addParseDataTask(String source, String dataType) {
        executor.execute(new GetDataTask(source, dataType, stats));
    }

    // here's the method that I need
    public String getInfo() {
        StringBuilder info = new StringBuilder();
        //and here's the method that I'm missing - executor.getActiveThreads()

        // >>> iterate "stats"'s values to build the info string ...            

        return info.append(executor.toString()).toString();
   }
}

UPDATE

You can easily change that approach to pulling the info by iterating the Map's keys (which are the executing tasks) and call toString on them. This is quite similar to saka's approach, though. Maybe you feel more comfortable with his.

Answer 4

Since you have control over the used executor, I would use the ThreadPoolExecutor's beforeExecute and afterExecute methods to keep track of running tasks and use that to create a getActiveTasks method.

import java.util.Set;
import java.util.concurrent.*;

public class ActiveTasksThreadPool extends ThreadPoolExecutor {

    private final ConcurrentHashMap<Runnable, Boolean> activeTasks = new ConcurrentHashMap<>();

    public ActiveTasksThreadPool(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
    }

    @Override
    protected void beforeExecute(Thread t, Runnable r) {

        activeTasks.put(r, Boolean.TRUE);
        super.beforeExecute(t, r);
    }

    @Override
    protected void afterExecute(Runnable r, Throwable t) {

        super.afterExecute(r, t);
        activeTasks.remove(r);
    }

    public Set<Runnable> getActiveTasks() {
        // the returned set will not throw a ConcurrentModificationException.
        return activeTasks.keySet();
    }

    public static void main(String[] args) {

        final int maxTasks = 5;
        ActiveTasksThreadPool tp = new ActiveTasksThreadPool(maxTasks, maxTasks, 10, TimeUnit.SECONDS, new SynchronousQueue<Runnable>());
        try {
            System.out.println("Active tasks: " + tp.getActiveTasks());
            final CountDownLatch latch = new CountDownLatch(1); 
            for (int i = 0; i < maxTasks; i ++) {
                final int rnumber = i;
                tp.execute(new Runnable() {
                    @Override
                    public void run() {
                        try { latch.await(); } catch (Exception e) {
                            e.printStackTrace();
                        }
                    }
                    @Override
                    public String toString() {
                        return "Runnable " + rnumber;
                    }
                });
            }
            Thread.sleep(100L); // give threads a chance to start
            System.out.println("Active tasks: " + tp.getActiveTasks());
            latch.countDown();
            Thread.sleep(100L); // give threads a chance to finish
            System.out.println("Active tasks: " + tp.getActiveTasks());
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            tp.shutdownNow();
        }
    }

}

Answer 5

You just need to store the references to the running threads somewhere which will be triggered within the ThreadPoolExecutor, adding on top of the other answers, this is an example of a small application which reads Thread states running inside the ThreadPoolExecutor every 1 second until shutdown:

package sample;

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Random;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class Test {

    public static void main(String[] args) {
        ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(10);

        for (int i = 1; i <= 10; i++)
        {
            Task task = new Task("Task " + i);
            executor.execute(task);
        }

        executor.shutdown();

        try {
            while (!executor.awaitTermination(1, TimeUnit.SECONDS)) {
                System.out.println("Awaiting completion of threads, threads states: " + Task.getThreadsStateCount());
            }

        } catch (InterruptedException e) {
        }

        System.out.println("Executor shutdown -> " + executor.isShutdown());
    }
}

class Task implements Runnable {

    static final List<Thread> activeTasks = Collections.synchronizedList(new ArrayList<>());
    static final Random r = new Random();

    private String name;

    public Task(String name) {
        this.name = name;
    }

    @Override
    public void run() {
        Thread t = Thread.currentThread();
        System.out.println("current thread : " + t.getName() + " group " + t.getThreadGroup() + " state " + t.getState());
        activeTasks.add(t);

        try {
            int tries = 0;

            while (tries < 10) {
                int randomNum = r.nextInt(10000);
                // do some expensive computation
                for(int i = 0; i < 4; i++) {
                    isPrime(r.nextLong());
                }

                // now sleep
                Thread.sleep(randomNum);
                tries++;
            }

        } catch (InterruptedException e) {
        }

        System.out.println("completed task for thread : " + t.getName() + " group " + t.getThreadGroup() + " state " + t.getState());
    }

    static boolean isPrime(long n)
    {
        if (n <= 1)
            return false;
        if (n <= 3)
            return true;

        if (n % 2 == 0 || n % 3 == 0)
            return false;

        for (int i = 5; i * i <= n; i = i + 6)
            if (n % i == 0 || n % (i + 2) == 0)
                return false;

        return true;
    }

    public static String getThreadsStateCount() {
        return "NEW: " + getCountThreadsState(Thread.State.NEW) +
                " ,RUNNABLE: " + getCountThreadsState(Thread.State.RUNNABLE) +
                " ,WAITING: " + getCountThreadsState(Thread.State.WAITING) +
                " ,TIMED_WAITING: " + getCountThreadsState(Thread.State.TIMED_WAITING) +
                " ,BLOCKED: " + getCountThreadsState(Thread.State.BLOCKED) +
                " ,TERMINATED: " + getCountThreadsState(Thread.State.TERMINATED);
    }

    public static long getCountThreadsState(Thread.State state) {
        return activeTasks.stream().filter(x -> x.getState() == state).count();
    }
}

// prints something like:

Awaiting completion of threads, threads states: NEW: 0 ,RUNNABLE: 1 ,WAITING: 0 ,TIMED_WAITING: 9 ,BLOCKED: 0 ,TERMINATED: 0