2

I have a ConcurrentLinkedQueue and I want to split it into two halves and let two separate threads handle each. I have tried using Spliterator but I do not understand how to get the partitioned queues.

ConcurrentLinkedQueue<int[]> q = // contains a large number of elements
Spliterator<int[]> p1 = q.spliterator();
Spliterator<int[]> p2 = p1.trySplit();
p1.getQueue(); 
p2.getQueue();

I want to but cannot do p1.getQueue() etc.

Please let me know the correct way to do it.

a_fan
  • 383
  • 2
  • 22

1 Answers1

4

You can't split it in half in general, I mean to split in half this queue must have a size at each point in time. And while CLQ does have a size() method, it's documentation is pretty clear that this size requires O(n) traversal time and because this is a concurrent queue it's size might not be accurate at all (it is named concurrent for a reason after all). The current Spliterator from CLQ splits it in batches from what I can see.

If you want to split it in half logically and process the elements, then I would suggest moving to some Blocking implementation that has a drainTo method, this way you could drain the elements to an ArrayList for example, that will split much better (half, then half again and so on).

On a side note, why would you want to do the processing in different threads yourself? This seems very counter-intuitive, the Spliterator is designed to work for parallel streams. Calling trySplit once is probably not even enough - you have to call it until it returns null... Either way doing these things on your own sounds like a very bad idea to me.

Eugene
  • 117,005
  • 15
  • 201
  • 306
  • 1
    You don’t have to call it until it returns `null`. The logic is more complicated. But calling once is indeed not enough, especially when the source is a `ConcurrentLinkedQueue`. – Holger Nov 24 '17 at 14:55
  • @Holger I assume not enough has to do with the fact that this is `concurrent` structure, right? – Eugene Nov 24 '17 at 16:26
  • 2
    You already described the problem very well, it’s concurrent and it’s linked, so there is no easy way to split off the “right” number of elements, so the implementation starts with a tiny buffer for the first split, raising the buffer size on each subsequent split, so for a really large `ConcurrentLinkedQueue` you need quiet a number of `trySplit` calls before you get anywhere near the true half. – Holger Nov 24 '17 at 16:36