Downloading algorithm

Question

I have this weird question. I want to create a down-loader tool. The idea i have in mind is, while downloading any file from the source i should be able to divide the file in chunks. for example. if the size of file is 100 MB then i want 5 streams pointing to my machine to download 20MB each simultaneously so that makes 5*20=100. the solution i have for this problem is that for my client down-loader tool there will be a server. first the file will be downloaded on cloud server(coz speed is very fast will take seconds). and then from my server i can get as many streams as i want depending on size of file. This idea will make me utilize the full bandwidth of the connection.

I'm using java btw!!!

You just stated what you want to do. That's great. Sounds interesting. SO is for asking specific development questions. Do you have any? — MadConan, Jun 25 '15 at 12:25

score 2 · Answer 1 · answered Jun 25 '15 at 12:36

2

If the original location of the file is on a slow server, downloading it to a cloud server won't be fast.

When it's on the cloud server, it won't be any faster to download it in separate chunks than in one chunk.

Therefore your idea doesn't really work, except in a case where for some reason the cloud server would be able to download the file faster than you directly.

That's just not how networking works.

answered Jun 25 '15 at 12:36

Kayaman

72,141
5
83
121

So what do you suggest? can you list some downloading algorithms? The one i know is prediction algorithm and the one i stated.. There are ways to speed up the downloading.. I'm after one of those.. suggest if you have something to say.. – Bajinder Budhwar Jun 26 '15 at 13:54
I recommend you read up on networking. Then you'll realize why your idea doesn't work, and maybe learn something new. – Kayaman Jun 29 '15 at 05:07

score 0 · Answer 2 · answered Jul 10 '15 at 01:59

Well.. @kayaman I did some digging about the question i asked. and found that most of the servers allow to skip the bytes and multiple connection to the data. SO if i want to download a file of size 80MB then i can get 8 connection pointing to the same file on server and each can help me download the 80/8=10MB. to be more clear.. for example.. if byte rage is from 0-80 (which weight total of 80MB)..

connection 1 -- 0-10 MB.

connection 2 -- 11-20 MB.

....

connection 8 -- 71-80 MB

And HTTP also support this with range attribute in header.. There should be a check in downloader for this if server allows multiple connection to the file to be safer side.. "But Most does"..

I was just getting the idea wrong way "file server to my server and then multiple connections to the client". Its just that most of file server already have the functionality inbuilt.. And to other thing that you pointed regarding can't increase the bandwidth window.. That's true.. But if you download a file linearly with one connection its slow than the file downloaded in chunks. i don't know why but this works.. :p

Thanks,

Downloading algorithm

2 Answers2