Processing all elements in a BlockingQueue

Question

I am parsing an XML document, building an object from the information that I parse, and then putting that object into a blocking queue.

For testing purposes, the size of the blocking queue is 3 and I have 4 pages that I need to process. What I'd like to happen is that after all possible objects have been added they should all be taken off. I've tried using a while loop (as shown below) to take all the elements out of the queue but a there seems to be a null preventing it from going through the rest of the queue.

I'm new to threads in Java and I'm guessing that's where my trouble lies.

Any help would be appreciated.

Runnable parser = new Runnable()
{
    public void run()
    {
        try
        {
            saxParser.parse("file_to_parse.xml", handler);
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
};

Runnable takeOff = new Runnable()
{
    public void run()
    {
        try
        {
            while(handler.bq.peek() != null)
            {
                handler.bq.take();
            }
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
};

new Thread(parser).start();
new Thread(takeOff).start();

score 4 · Accepted Answer · answered Jun 24 '13 at 16:06

One problem is that take is going to block on an empty queue (poll is the method that returns null on an empty queue), so your loop won't terminate (unless parser is adding a null to the queue, which ought to be an error). You're also calling take too often - the first take call (in the while loop's guard) is removing an element, and then the second take call (inside the while loop) is removing another element, which means that you're discarding half of the queue's elements. Try something like this instead

try {
    String temp;
    while((temp = handler.bq.poll()) != null) {
        // process temp
    }
}

This will only work if you don't start takeOff until parser is finished adding items to the queue (when offer returns false) - otherwise poll might return null and terminate takeOff while parser is still adding items to the queue. Here's an alternative that lets you run parser and takeOff at the same time

String temp;
try {
    while(true) {
        temp = handler.bq.take();
        // process temp
    }
catch (InterruptedException ex) {
    // ignore exception
}
while((temp = handler.bq.poll()) != null) {
    // process temp
}

Then you need to interrupt() the takeOff thread when parser is finished: this will exit the while(true) loop, after which the thread will go to the while((temp = handler.bq.poll()) != null) loop to finish removing items from the queue.

Thanks for the response. I should note that when I said `while(handler.bq.take() != null)` that was a mistake. It was supposed to be `peek()`; I've updated my original post. I will work with your suggestions and hopefully get something working. Thanks again. — joshft91, Jun 24 '13 at 16:15
Out of curiosity, why do you use a `String temp` and what do you mean by `// process tmep`? — joshft91, Jun 24 '13 at 16:27
@joshft91 I was assuming that the blockingqueue held strings - if it holds something else e.g. `XMLDocuments` then this should be `XMLDocument temp;` `// process temp` means to do whatever it is you're supposed to be doing with `temp` - if all you're doing is removing items from the queue without processing them in any way, then this should be a nop. Also, `while(handler.bq.peek() != null)` has the same problem as `while(handler.bq.poll() != null)` in that it might prematurely terminate the thread. — Zim-Zam O'Pootertoot, Jun 24 '13 at 16:35
Ah, alright. Yeah, the blockingqueue is holding objects and all I'm doing with them right now is removing/discarding the objects. — joshft91, Jun 24 '13 at 16:37

Processing all elements in a BlockingQueue

1 Answers1