2

I was downloading a 200MB file yesterday with FlashGet in the statistics it showed that it was using the HTTP1.1 protocol.
I was under the impression that HTTP is a request-response protocol and most generally used for web pages weighing a few KiB...I don't quite understand how it can download MB's or GB's of data and that too simultaneously through 5(or more) different streams.

tshepang
  • 12,111
  • 21
  • 91
  • 136
Kevin Boyd
  • 12,121
  • 28
  • 86
  • 128

3 Answers3

6

HTTP/1.1 has a "Range" header that can specify what part of a file to transfer over the connection. The download manager can make multiple connections, specifying different ranges to transfer. It would then combine the chunks together to build the full file.

Jeffrey Bird
  • 76
  • 2
  • If the connection breaks or dies, how does it know where to start from again? How does it keep track of bytes transferred etc in this case? – Kevin Boyd Sep 30 '09 at 06:29
  • 1
    The downloader would know how much data had been transferred on a given connection just by counting how many bytes it had received. It could then make a new connection and specify a range of bytes to transfer that starts after the transferred bytes. The download process would basically be: 1. retrieve the length of the data file 2. calculate a nunber of chunks to transfer e.g. break the file into say 1MB chunks 3. Start a number of parallel connections that transfer these chunks i.e. the chunks would be say bytes 1-1000000, 1000001-2000000, and so on. – Jeffrey Bird Sep 30 '09 at 08:38
1

There is no size limit in http. It is used for web pages, but it is also used to deliver a huge majority of the content on the Internet. It's more a matter of bandwidth that limits sizes, not the protocol itself. And of course, this was more of a limit in the early days. (and, I suppose, those still on dial-up)

David
  • 72,686
  • 18
  • 132
  • 173
  • What process does the download manager use to download huge files with multiple streams with just a handful of requests? – Kevin Boyd Sep 30 '09 at 04:20
0

These links might help:

HTTP

HTTP Persistent Connections

Chunked Transfer Encoding

Zaki
  • 6,997
  • 6
  • 37
  • 53
  • Incidentally I had read them all 2 days ago however I still could not understand the basics of Download Managers and HTTP 1.1 ... – Kevin Boyd Sep 30 '09 at 08:27