2

I'm using Scala's FS2 stream library.

I have a Stream[F, [Stream[F, A]] where both the inner streams and the outer stream are infinite (with appropriate Async instances for F). I want to end up with a Stream[F, A] that concurrently pulls from the outer stream as well as an inner stream, where every new element from the outer stream replaces the current inner stream that I'm pulling from. In particular I want to "eventually" get to at least attempt to pull (although I may be interrupted by the outer stream before I can do so) from all of the inner streams.

My attempts to do this with an external Async reference that holds the current inner stream seem to end in Interrupt exceptions getting thrown.

I do not want a simple flatMap or even concurrent.join. Because my inner streams are infinite, these will never get to more than a finite number of inner streams.

Is there a way of achieving this with FS2?

badcook
  • 3,699
  • 14
  • 25
  • 1
    Sounds a lot like `Observable#switchMap` from Monix. I've [seen](https://gist.github.com/mpilquist/9deef315f0aafc0de9b6756fe4146231) FS2 implementation of that. Never used it tho. – Oleg Pyzhcov Oct 04 '17 at 18:55
  • 1
    There is a method I know from proofs that is called [dove-tailing](https://en.wikipedia.org/wiki/Dovetailing_(computer_science)). Given your setup of (countably) infinitely many infinite streams, it would ensure that all streams get eventually pulled; and moreover, for any element of any inner stream, there would be a finite time when it will be pulled. Is this what you want? Because I don't understand your description. – ziggystar Oct 05 '17 at 07:55
  • @OlegPyzhcov you've got it right (I would accept that as an answer). – badcook Oct 05 '17 at 13:38
  • @ziggystar dovetailing is interesting as a proof technique but too slow to be practical. I don't need every element of every stream; I'm happy to throw away the current inner stream I'm looking at when a new one comes in. – badcook Oct 05 '17 at 13:39
  • The author of fs2 has shared a `switchMap` implementation here: https://gist.github.com/mpilquist/9deef315f0aafc0de9b6756fe4146231 – Rajit May 18 '18 at 22:22

0 Answers0