Nonblocking DelayQueue, Java

Question

I'm using Java's DelayQueue to dispatch events after a second delay. But the problem is that, under heavy load, my consumers on DelayQueue block until considerable bulk of offer() operations from another threads are gone.

Does anyone know non-blocking delay queue implementations in Java?

When you say "bulk" you mean you are doing a lot of offer calls, I suppose. How many? — rghome, Feb 06 '15 at 14:55

score 2 · Answer 1 · edited May 23 '17 at 12:13

I think you are misunderstanding either the DelayQueue API or the way that thread scheduling works.

If you want genuine non-blocking queue operations, then DelayQueue already provides them. For example poll() will immediately return either a queue entry or null. It won't block the calling thread. The offer(...) method is the equivalent method for inserting without blocking.

On the other hand, if you are actually saying that some threads are being "starved", then there is not much you can do about that. Java thread scheduling is not "fair":

If you have lots of threads that are runnable, no attempt is made to give each one a similar amount of run time.
If you have multiple threads waiting on a primitive lock or notify, then the scheduler won't make any attempt to pick one thread "fairly".

Thread starvation is going to be more likely if you have lots more threads than you have cores to run them.

The best solution is to design your algorithms such that it doesn't matter if threads are scheduled unfairly. It shouldn't matter; see Is a DelayQueue without fairness problematic?.

For the record, I'm not aware of a DelayQueue replacement that advertises fair scheduling.

Thanks for the reply. I took a look at implementation of DelayQueue and it turned out that it uses ReentrantLock to poll() and add() elements.That's why I cannot write and read at the same time when the load is immense. — Anton K., Feb 09 '15 at 11:19
Ah ... I see what you meant in your question now. You are talking about contention rather than indefinite blocking. — Stephen C, Feb 09 '15 at 11:36

score 0 · Accepted Answer · answered Feb 09 '15 at 11:18

Unfortunately, DelayQueue is blocking queue and it doesn't return immediately if it's intensely being written to because it uses a lock.

public E poll() {
    final ReentrantLock lock = this.lock;
    lock.lock();
    try {
        E first = q.peek();
        if (first == null || first.getDelay(NANOSECONDS) > 0)
            return null;
        else
            return q.poll();
    } finally {
        lock.unlock();
    }
}

So if many threads write to it, as Stephen said, there is not much you can do about that.

I solved the problem by using ConcurrentSkipListSet with DelayedElement.

public class DelayedElement implements Comparable<DelayedElement> {

private final Long initTime;
private final String msgId;

public DelayedElement(Long initTime, String msgId) {
    this.initTime = initTime;
    this.msgId = msgId;
}             

@Override
public int hashCode() {
    int hash = 5;
    hash = 29 * hash + Objects.hashCode(this.initTime);
    hash = 29 * hash + Objects.hashCode(this.msgId);
    return hash;
}

@Override
public boolean equals(Object obj) {
    if (obj == null) {
        return false;
    }
    if (getClass() != obj.getClass()) {
        return false;
    }
    final DelayedElement other = (DelayedElement) obj;
    if (!Objects.equals(this.initTime, other.initTime)) {
        return false;
    }
    if (!Objects.equals(this.msgId, other.msgId)) {
        return false;
    }
    return true;
}
@Override
public int compareTo(DelayedElement o) {
    return -o.initTime.compareTo(initTime);
    }
}

In my producers' threads, I add each element with a second's delay. In my consumer thread, I simply read elements which has a second's delay like:

 long diff = System.nanoTime() - TimeUnit.MILLISECONDS.toNanos(1000L);
 NavigableSet<DelayedElement> set = queue.headSet(
 new DelayedElement(diff, "", null));
 //further processing goes here

This way I achieve non-blocking nature and can safely write and read from the Collection at full throttle.

Nonblocking DelayQueue, Java

2 Answers2