1

How do I provide a reactivestreams api for a db that does not support streaming? Like lets say for example dynamodb. When doing a get call, dynamodb is going to return all the results. So even if I wrap the get call in a Source, How do I handle backpressure from the downstream stages? Also how do I implement write calls into db? What will my sink look like? Any pointers on this will be helpful.

Aravindh S
  • 1,185
  • 11
  • 19

1 Answers1

0

One option is to implement your database Source using an ActorPublisher -

See: http://doc.akka.io/docs/akka/2.4.11/scala/stream/stream-integrations.html#ActorPublisher

Just mixing in this trait and implementing the command interface will give you a reactive streams compliant data publisher that can handle down stream backpressure. Your publisher will get a Request message if subscribers down stream pull more data and it will have access to the current perceived demand if it needs to actively push more data down stream. You can then plug this publisher into your Akka Streams pipeline by creating a Source from it:

Source.actorPublisher[Data](MyPublisher.props).runWith(MySink)

To deal with the fact that the underlying DB is itself NOT reactive, you would need to implement some buffering and polling logic within the ActorPublisher.

simonl
  • 1,240
  • 7
  • 19