0

I use youtube-dl to obtain the URL of the binary stream of a YouTube video, that for whatever reason i'd like to download, i do it like this:

youtube-dl -f bestaudio -g --skip-download https://www.youtube.com/watch?v=nk5YtLYcH74

After i have the URL, I curl it, somehow like this:

https://r6---sn-gqn-vhgl.googlevideo.com/videoplayback\?id\=afe82f21e356....

or i could forward the output into a file of course, but this is just to explain. I'd like to download it, but not with curl, but with a Scala HTTP client, for now akka-http. But that way i get:

IllegalResponseException: Response Content-Length 195022703 exceeds the configured limit of 8388608

I know this is a large stream, but how does curl do it automatically? How does it request chunks, or how does it work, how should i write it in Scala/Java?

Some code and more:

val connectionFlow = Http().outgoingConnectionTls(hostname)
val responseFuture =
  Source.single(HttpRequest(uri = path, method = GET))
    .via(connectionFlow)
    .runWith(Sink.foreach[HttpResponse] { r =>
      logger.info(s"${r.status}")
  })
Hunor Kovács
  • 1,062
  • 9
  • 16

1 Answers1

1

The default content-length is 8M for akka-http.

Increase the max-content-length value for the akka http client parsing in the application.conf file and it should work fine.

akka.http.client.parsing.max-content-length =78m

more about the conf and info on this link http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/scala/http/configuration.html

Rjk
  • 1,356
  • 3
  • 15
  • 25
  • Thanks, that just killed my question :)) I actually wanted to process chunks and to the utility to define in how large packets should the data come, and in a reactive backpressured stream. I'll continue trying to figure it out. – Hunor Kovács Jul 27 '15 at 11:14