Streaming Chunked HTTP Entities with vert.x

Question

I'm beginning an initial review of vert.x and comparing it to akka-http. One area where akka appears to shine is streaming of response bodies.

In akka-http it is possible to create a streaming entity that utilizes back-pressure which allows the client to decide when it is ready to consume data.

As an example, it is possible to create a response with an entity consisting of 1 billion instances of "42" values:

//Iterator is "lazy", therefore this function returns immediately
val bodyData : () => Iterator[ChunkStreamPart] = () => 
  Iterator 
    .continually("42")
    .take(1000000000)
    .map(ChunkStreamPart.apply)

val route = 
  get {

    val entity : HttpEntity = 
      Chunked(ContentTypes.`text/plain(UTF-8)`, Source fromIterator bodyData)

    complete(HttpResponse(entity=entity))
  }

The above code will not "blow up" the server's memory and will return the response to the client before the billion values have been generated.

The "42" values will get created on-the-fly as the client tries to consume the response body.

Question: is this streaming capability also present in vert.x?

A cursory review of the HttpServerResponse class would indicate that it is not since the write member function can only take in a String or a vert.x Buffer. From my limited understanding it seems that Buffer is not lazy and holds the data in memory which means the 1 billion "42" example would crash a server with just a few concurrent requests.

Thank you in advance for your consideration and response.

First thing that pops up in google: [Vert.x Reactive Streams Integrations](http://vertx.io/docs/vertx-reactive-streams/java/). — pedromss, Sep 05 '17 at 13:38
@pedromss I'm familiar with the page, the notion of a `Pump` seems to indicate that the `Buffer` would fill up with the billion `"42"` values in memory... — Ramón J Romero y Vigil, Sep 05 '17 at 13:43
Are you familiar with the concept of [Reactive Streams](http://www.reactive-streams.org/)? That's the concept of that vert.x reactive streams implements so it should answer your question, no? :) Either that or I'm not understanding the question :( — pedromss, Sep 05 '17 at 13:46
@pedromss Yes, I'm quite familiar with reactive streams. It's possible that `vert.x` implemented reactive streams for connection handling but not entity handling. This is at the heart of my question: how do I construct a response entity which is a reactive stream??? — Ramón J Romero y Vigil, Sep 05 '17 at 13:50
May I ask: This looks like a route exposed through HTTP, what happens when you do a GET? (using akka-http). Do you get a bunch for '42's every now and then? — pedromss, Sep 05 '17 at 14:20
When you do a get the response comes back immediately. If the client begins to read the body then 42s get generated as the client reads them, if the client sleeps for 5 minutes before reading the body then the server performs no work during that time. — Ramón J Romero y Vigil, Sep 05 '17 at 14:25
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/153698/discussion-between-pedromss-and-ramon-j-romero-y-vigil). — pedromss, Sep 05 '17 at 14:49
did you get an answer? I am at the same point at the moment :-D — Auryn, Jun 05 '19 at 19:44

Streaming Chunked HTTP Entities with vert.x

0 Answers0