How to use an Akka Streams SourceQueue with PlayFramework

Question

I would like to use a SourceQueue to push elements dynamically into an Akka Stream source. Play controller needs a Source to be able to stream a result using the chuncked method.
As Play uses its own Akka Stream Sink under the hood, I can't materialize the source queue myself using a Sink because the source would be consumed before it's used by the chunked method (except if I use the following hack).

I'm able to make it work if I pre-materialize the source queue using a reactive-streams publisher, but it's a kind of 'dirty hack' :

def sourceQueueAction = Action{

    val (queue, pub) = Source.queue[String](10, OverflowStrategy.fail).toMat(Sink.asPublisher(false))(Keep.both).run()

    //stupid example to push elements dynamically
    val tick = Source.tick(0 second, 1 second, "tick")
    tick.runForeach(t => queue.offer(t))

    Ok.chunked(Source.fromPublisher(pub))
  }

Is there a simpler way to use an Akka Streams SourceQueue with PlayFramework?

Thanks

I quite like this approach. Why did you find it to be "dirty"? — akauppi, Apr 04 '18 at 07:28

Loic · Accepted Answer · 2016-04-07T11:24:08.873

27

The solution is to use mapMaterializedValue on the source to get a future of its queue materialization :

def sourceQueueAction = Action {
    val (queueSource, futureQueue) = peekMatValue(Source.queue[String](10, OverflowStrategy.fail))

    futureQueue.map { queue =>
      Source.tick(0.second, 1.second, "tick")
        .runForeach (t => queue.offer(t))
    }
    Ok.chunked(queueSource)

  }

  //T is the source type, here String
  //M is the materialization type, here a SourceQueue[String]
  def peekMatValue[T, M](src: Source[T, M]): (Source[T, M], Future[M]) = {
    val p = Promise[M]
    val s = src.mapMaterializedValue { m =>
      p.trySuccess(m)
      m
    }
    (s, p.future)
  }

edited Apr 07 '16 at 11:24

answered Apr 07 '16 at 05:12

Loic

3,310
4
25
43

Why if I do `queueSource.map { _.toUpperCase }` for example I don't get a Source[String,NotUsed]? Instead this returns the error `Expression of type queueSource.Repr[String] doesn't conform to expected type Source[String,NotUsed].` Where do you'll do transformations to the source's elements? Like ticks in [your example](http://loicdescotte.github.io/posts/play-akka-streams-queue/) – gabrielgiussi Feb 07 '17 at 02:12
Can you do this in Java? – El Mac Apr 30 '18 at 09:17
1

[Source.preMaterialize](https://doc.akka.io/api/akka/current/akka/stream/scaladsl/Source.html#preMaterialize()(implicitmaterializer:akka.stream.Materializer):(Mat,Source.this.ReprMat[Out,akka.NotUsed])) can also be used instead of the `peekMatValue` method – Roel Van der Paal Nov 02 '18 at 13:57

score 1 · Answer 2 · answered Mar 27 '18 at 18:47

1

Would like to share an insight I got today, though it may not be appropriate to your case with Play.

Instead of thinking of a Source to trigger, one can often turn the problem upside down and provide a Sink to the function that does the sourcing.

In such a case, the Sink would be the "recipe" (non-materialized) stage and we can now use Source.queue and materialize it right away. Got queue. Got the flow that it runs.

answered Mar 27 '18 at 18:47

akauppi

17,018
15
95
120

interesting, I would be happy to see an example :) – Loic Mar 29 '18 at 07:02
@Loic I would, but the code I have using it is currently closed source. And with some more days thinking about it (low flame), I think it's same as making the SourceQueue at the upper levels, and exposing the 'offer' function to whatever one wants to pump the source with. – akauppi Mar 30 '18 at 12:00

How to use an Akka Streams SourceQueue with PlayFramework

2 Answers2

Linked