0

The getQueue() method provides access to the underlying blocking queue in the ThreadPoolExecutor, but this does not seem to be safe.

A traversal over the queue returned by this function might miss updates made to the queue by the ThreadPoolExecutor.

"Method getQueue() allows access to the work queue for purposes of monitoring and debugging. Use of this method for any other purpose is strongly discouraged."

What would you do if you wanted to traverse the workQueue used by the ThreadPoolExecutor? Or is there an alternate approach?

This is a continuation of.. Choosing a data structure for a variant of producer consumer problem

Now, I am trying the multiple producer multiple consumer, but I want to use some existing threadpool, since I don't want to manage the threadpool myself, and also I want a callback when ThreadPoolExecutor has finished executing some task alongwith the ability to examine in a thread safe way the "inprogress transactions" data structure.

Community
  • 1
  • 1
Abhijeet Kashnia
  • 12,290
  • 8
  • 38
  • 50

3 Answers3

1

You can override the beforeExecute and afterExecute methods to let you know that a task has started and finished. You can override execute() to know when a task is added.

The problem you have is that the Queue is not designed to be queried and a task can be consumed before you see it. One way around this is to create you own implementation of a Queue (perhaps overriding/wrapping a ConcurrentLinkedQueue)

BTW: The queue is thread-safe, however it is not guaranteed you will see every entry.

A ConcurrentLinkedQueue.iterator() is documented as

Returns an iterator over the elements in this queue in proper sequence. The returned iterator is a "weakly consistent" iterator that will never throw ConcurrentModificationException, and guarantees to traverse elements as they existed upon construction of the iterator, and may (but is not guaranteed to) reflect any modifications subsequent to construction.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • I'm ok with taking a snapshot at that point of time, like Collections.unmodifiable(), but I need to know while I take that snapshot,the queue will not change. So, I need to lock the queue on something, but the ThreadPoolExecutor will not respect my lock constraints. Are you suggesting that I create my own CustomBlockingqueue, that I pass to the ThreadPoolExecutor? I don't understand how that will help, please elaborate. – Abhijeet Kashnia May 09 '11 at 10:30
  • Can Collections.unmodifiable() be considered as an atomic operation? Because, I think only atomic operations will be threadsafe. – Abhijeet Kashnia May 09 '11 at 10:35
  • 2
    The problem you have is that ConcurrentLinkedQueue is designed for concurrent/lockless queuing. ThreadPoolExecutor has no say in the matter. You can give it a Queue which behaves as you want. – Peter Lawrey May 09 '11 at 10:38
  • 2
    You cannot use `Collections.unmodifiableXxxx()` with queues. It only wraps the underlying collections and does not copy it. Atomic operations are easier to make thread safe, however you can make Iterator thread safe by locking the collection first. (assuming the collection honours locks) – Peter Lawrey May 09 '11 at 10:40
  • Could you please also expand on the statement "The queue is thread-safe, however it is not guaranteed you will see every entry." I think thread safety implies that every entry be seen. – Abhijeet Kashnia May 10 '11 at 10:44
  • Thread safety means it is guaranteed to behave as documented when modified by multiple threads. Collections which are not thread safe, are not guaranteed to behave as documented when modified by multiple threads. (They don't even guarantee you will see a ConcurrentModificationException) The problem you have is that you want it to behave differently to how it is documented. You can't do this without changing the behaviour of the queue. – Peter Lawrey May 10 '11 at 11:07
1

If you wish to copy the items in the queue and ensure that what you have in the queue has not been executed, you might try this:

a) Introduce the ability to pause and resume execution. See: http://download.oracle.com/javase/1,5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html

b) first pause the queue, then copy the queue, then resume the queue.

And then i have my own question. The problem i see is that while you execute your "Runnable", that "Runnable" is not placed in the queue, but a FutureTask "wrapper", and i cannot find any way to determine just which one of my runnables i'm looking at. So, grabbing and examining the queue is pretty useless. Does anybody know aht i missed there?

Paul G
  • 21
  • 1
  • Thanks Paul, the last paragraph of you answer makes complete sense, as I have realised that I need to maintain my runnables in a separate data structure. – Abhijeet Kashnia Jul 05 '11 at 20:10
0

If you are following Jon Skeet's advice in your accepted answer from your previous question, then you'll be controlling access to your queues via locks. If you acquire a lock on the in-progress queue then you can guarantee that a traversal will not miss any items in it.

The problem with this of course is that while you are doing the traverse all other operations on the queue (other producers and consumers trying to access it) will block, which could have a pretty dire effect on performance.

Community
  • 1
  • 1
Qwerky
  • 18,217
  • 6
  • 44
  • 80
  • Thanks for the response. Yes, I am trying to follow his advice, with the additional use of threadExecutor to handle the thread pool management for me. I think I could do it on my own without ThreadPoolExecutor, but then I'll have to write extra code to handle the threadpool, and callbacks on completion of the task, which I'm trying to avoid. – Abhijeet Kashnia May 09 '11 at 10:32