0

I am trying to stream results from a query using Slick within a Java application, using Akka Streams and Postgres:

Source mySource = Slick.source(
  slickSession,
  "SELECT * from message where started_instant is null",
  (this::createQueueItemFromSlickRow));

In the Slick documentation it is clear that in order to stream with Postgres, additional parameters need to be set. However, I can only find Scala examples, and there is no information in the Alpakka documentation on how to configure this.

As it is, I can't process any new objects that come into the database after the stream has started.

Jeffrey Chung
  • 19,319
  • 8
  • 34
  • 54
Noremac
  • 3,445
  • 5
  • 34
  • 62

1 Answers1

1

As you stated, the Slick documentation notes the following:

Note: Some database systems may require session parameters to be set in a certain way to support streaming without caching all data at once in memory on the client side. For example, PostgreSQL requires both .withStatementParameters(rsType = ResultSetType.ForwardOnly, rsConcurrency = ResultSetConcurrency.ReadOnly, fetchSize = n) (with the desired page size n) and .transactionally for proper streaming.

Alpakka's Slick support recently exposed the PreparedStatement in its Java DSL1, which you can use to set the aforementioned parameters if you cannot use the Scala API. For example:

PreparedStatement statement =
  connection.prepareStatement(
    "MY SQL STATEMENT",
    ResultSet.TYPE_FORWARD_ONLY, // java.sql.ResultSet
    ResultSet.CONCUR_READ_ONLY);

Check out the SlickTest.java class and the above pull request for more details.

1This change is part of the upcoming Alpakka 2.0.2 release. (Version 2.0.1 is the current release at the time of this writing.)

Jeffrey Chung
  • 19,319
  • 8
  • 34
  • 54