0

I'm searching if it's possible to use Java stream with queue/backpressure. Something like:

iterator -> (queue?) -> parallelstream -> map -> collector

Mapping is slower than iterator (even if multithreaded), so iterator should be read a slower pace (backpressure) or data from iterator should be queued to wait for mapping. I've found that reactive stream can be used for backpressure (https://blog.softwaremill.com/how-not-to-use-reactive-streams-in-java-9-7a39ea9c2cb3) but what about a queue?

Gautham M
  • 4,816
  • 3
  • 15
  • 37
Marx
  • 804
  • 10
  • 23
  • 1
    Java stream API isn't reactive. So it doesn't support backpressure. It's a lightweight feature to create a data manipulation streams in a functional style. Consider RxJava or Project Reactor instead. – Ansar Ozden May 29 '21 at 00:40
  • Java 9 gained the [`Flow`](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/concurrent/Flow.html) interfaces as a façade for [reactive programming](https://en.wikipedia.org/wiki/Reactive_programming) implementations. See [*JEP 266: More Concurrency Updates*](http://openjdk.java.net/jeps/266). These interfaces correspond to the [*reactive-streams* specification](http://www.reactive-streams.org/). [*Reactive Streams*](https://en.wikipedia.org/wiki/Reactive_Streams) is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. – Basil Bourque May 29 '21 at 02:18
  • I've already sent a link to it, unfortunatelly it looks like this approach is still unfinished in java. What abut a queue inside stream? Couldn't google for good example – Marx May 29 '21 at 08:41
  • 1
    Streaming an iterator should already have a form of backpressure because the on-demand (pull) nature of it. Is it necessary to have a queue between the iterator and the parallelStream? Do you have a more concrete example code about what you try to achieve? – akarnokd May 29 '21 at 21:50
  • If an iterator is of unknown size, parallel stream parallelize only every 1024 items - that's the problem – Marx May 31 '21 at 04:45

0 Answers0