How to stream the response of dispatcher?

Question

I have to export the CSV data. The volume of data is very high. So I am streaming the response from the microservice. We hit our microservice using dispatcher.

def stream(method: String, urlString: String): Future[Source[ByteString, NotUsed]] =
    method match {
      case GET    =>
        val request = Http(url(urlString))
        request.map { response =>
          response.getStatusCode match {
            case StatusOk => Source.single(ByteString(response.getResponseBody))
          }
        }
    }

It will bring all the data. So to fix this issue, I like to modify it and streamed the data from here as well.

I searched a lot and found this question Scala dispatch stream response line by line

But it has no answer.

Thanks and any help will be appreciated.

score 0 · Accepted Answer · answered Aug 05 '19 at 06:54

After searching much, I read it as Input stream and converted to Akka Stream. It worked for me.

def stream(method: String, urlString: String): Future[Source[ByteString, Future[IOResult]]] =
    method match {
      case GET    =>
        val futureStream = Http(url(urlString) > as.Response(_.getResponseBodyAsStream))
        futureStream.map { inputStream =>
          val source = () => inputStream
          StreamConverters.fromInputStream(source)
        }
    }

How to stream the response of dispatcher?

1 Answers1