10

I found Slick 3.0 introduced a new feature called streaming

http://slick.typesafe.com/doc/3.0.0-RC1/database.html#streaming

I'm not familiar with Akka. streaming seems a lazy or async value, but it is not very clear for me to understand why it is useful, and when will it be useful..

Does anyone have ideas about this?

  • 1
    You should probably read up on Akka streams as I think this mechanism in slick is a good way to tie Slick and Akka streams together. With this mechanism, you can treat a database results collection as a Stream Source and then feed that into whatever processing pipeline/graph you want. Without this bridge, you'd have to write that part yourself, fully materializing the collection into memory first which may not be desired. – cmbaxter May 12 '15 at 12:57

1 Answers1

12

So lets imagine the following use case:

A "slow" client wants to get a large dataset from the server. The client sends a request to the server which loads all the data from the database, stores it in memory and then passes it down to the client.

And here we're faced with problems: The client handles the data not so fast as we wanted => we can't release the memory => this may result in an out of memory error.

Reactive streams solve this problem by using backpressure. We can wrap Slick's publisher around the Akka source and then "feed" it to the client via Akka HTTP.

The thing is that this backpressure is propagated through TCP via Akka HTTP down to the publisher that represents the database query.

That means that we only read from the database as fast as the client can consume the data.

P.S This just a little aspect where reactive streams can be applied.

You can find more information here:

Matthias Braun
  • 32,039
  • 22
  • 142
  • 171
Igor Mielientiev
  • 586
  • 4
  • 11