1

In my application I need to download automatically many files via HTTP, possibly quite large. I want to implement something like "batch" behavior, when all files that need to be downloaded are put into a queue, then several work threads pick up tasks from this queue and do the actual download in parallel in background. Also I need some usual behavior from download managers: handle incomplete or failed downloads, use temp folder/file name until the file is fully downloaded, etc. I tried googling, but I was surprised I did not find anything useful in Java to help in this task. The only things I've found so far were:

  1. Download manager in Java - No useful answers, just one link to

  2. http://luugiathuy.com/2011/03/download-manager-java/ - this one is better, but still very low level, too simple and does not cover 90% of my needs. Seems more like tutorial and less like library

  3. http://sourceforge.net/projects/jdm/ - despite its name, it's even more basic.

Does anybody know any other libraries that can implement such behavior?

Community
  • 1
  • 1
afrish
  • 3,167
  • 4
  • 30
  • 38

1 Answers1

0

The library that you can use is apache http components - it gives you a nice API for working with HTTP.

But you'd have to implement the rest yourself - batch downloading is effectively just a single download repeated many times. For that you'd need a separate library that implements. You can easily use the java 5 concurrency additions, e.g. Executors.newFixedThreadPool(..) to submit Runnables that perform the downloads.

I'd suspect there is no ready-to-use implementation, because it would be an end-user product, rather than a library. You'd have to gather the components yourself.

Bozho
  • 588,226
  • 146
  • 1,060
  • 1,140
  • 1
    I don't agree with you that this would be an end-user product. For me this seems like very common task. There are so many common things for any application that needs to download files using a queue. It would be pity if I had to implement this myself. Not that I don't like coding myself, I just don't like reinventing the wheel. HTTP Components is way to basic for this task, I need something more similar to traditional download manager, but without GUI. Let's see if somebody finds another solution. – afrish Jun 21 '13 at 14:53