0

I am trying to use boost asio in order to download 100 files in parallel, from a server with max speed 100kbps while my PC has 10mbps.

I decided against using threads due to the serious performance hit. I learned about select and I/O Multiplexing. I was pointed to this question: select functionality in boost::asio but the answer doesn't provide enough details.

In my application I use the sync_client example from here.

Please explain how to download 100 files in parallel in boost asio without threads, with some kind of select mechanism?

Community
  • 1
  • 1
Luka
  • 1,761
  • 2
  • 19
  • 30
  • Why would you do that? Downloading 100 files in parallel at 0.1 MB per second means that if the connection gets broken, none of the files will have finished. Why don't you just queue the 100 downloads, and download those files in smaller numbers instead? – Stephan Branczyk Jan 13 '14 at 04:21
  • I have permission from the server owner to do so many downloads. Also, I must download the files as fast as possible. – Luka Jan 13 '14 at 04:22
  • Are those files of tiny size to begin with? – Stephan Branczyk Jan 13 '14 at 04:23
  • Yes, very tiny (about 5kb) – Luka Jan 13 '14 at 04:23
  • When I say "as fast as possible" I mean, even 0.1 second slower matters. – Luka Jan 13 '14 at 04:24
  • 1
    'I decided against using threads due to the serious performance hit' - what performance hit? – Martin James Jan 13 '14 at 04:32
  • If one computer has 2, 4 or 8 cores and I start 100 threads I will really beat the crap out of it and make the CPU go 100%, not including context switching. I don't want the CPU go 100%, even for 1 second. – Luka Jan 13 '14 at 04:33
  • If the server capacity is 100 kbps then that's as fast as it gets. If you download 100 files simultaneously you'll get at most 1 kbps per download. And downloading small files over TCP/IP is extra slow because of the protocol. – molbdnilo Jan 13 '14 at 05:01
  • No, the server capacity is much higher than 10mbps, but every connection is limited to 100kbps. Is there a fastest but equally secure protocol for HTTP downloads than TCP? – Luka Jan 13 '14 at 05:03
  • 1
    @Luka I have BitTorrent running now. It is downloading stuff. It has 60 threads. CPU use 1%. 100 network threads is, like, nothing. OK, I would probably go with a pool of, say, 32, but only 'cos I have code for it already. – Martin James Jan 13 '14 at 05:16
  • Thanks for pointing it out... I will try to modify my threads code... – Luka Jan 13 '14 at 05:18
  • @Luka - Ooh, I didn't realise you already tried it with 100 threads. If you tried with 100 threads and you got 100% CPU use, you're doing it wrong somehow:) – Martin James Jan 13 '14 at 05:20
  • What's the fastest thread in c++, boost, std, posix? – Luka Jan 13 '14 at 05:24

0 Answers0