0

I am trying to write a simple download manager using python with concurrency. The aim is to use the Content-Length header from the response of a file url and splits the file into chunks then download those chunks concurrently. The idea works for all the urls which have Content-Length header but recently I came across some urls which doesn't serve a Content-Length header.

https://filesamples.com/samples/audio/mp3/sample3.mp3

HTTP/1.1 200 OK
Date:   Sat, 08 Aug 2020 11:53:15 GMT
Content-Type:   audio/mpeg
Transfer-Encoding:  chunked
Connection: close
Set-Cookie: __cfduid=d2a4be3535695af67cb7a7efe5add19bf1596887595; expires=Mon, 07-Sep-20 11:53:15 GMT; path=/; domain=.filesamples.com; HttpOnly; SameSite=Lax
Cache-Control:  public, max-age=86400
Display:    staticcontent_sol, staticcontent_sol
Etag:   W/"5def04f1-19d6dd-gzip"
Last-Modified:  Fri, 31 Jul 2020 21:52:34 GMT
Response:   200
Vary:   Accept-Encoding
Vary:   User-Agent,Origin,Accept-Encoding
X-Ezoic-Cdn:    Miss
X-Middleton-Display:    staticcontent_sol, staticcontent_sol
X-Middleton-Response:   200
CF-Cache-Status:    HIT
Age:    24
cf-request-id:  046f8413ab0000e047449da200000001
Expect-CT:  max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 5bf90932ae53e047-SEA

How can I get the content-length of the file without downloading the whole file?

Eka
  • 14,170
  • 38
  • 128
  • 212
  • 2
    Unfortunately you can't. If it is not provided by the server, there is not way to force the server to return that field in the header. – James Aug 08 '20 at 12:53
  • Is there any way to calculate it ourselves without downloading the whole files? – Eka Aug 08 '20 at 16:26
  • 1
    No. The information is simply not provided. The server opens a stream, but provides no info on how big the stream is. – James Aug 10 '20 at 12:54

0 Answers0