Multi Thread java downloader downloading corrupted file?

Question

I am attempting to make a multiThread downloader in java. It works but the file that I download is always missing some bytes.

I have searched and found many example of multiThreaded web crawlers but nothing simple, so can anyone tell me if my method could work?

I don't know if the problem is with ordering the bytes.

I have tried BlockingQueue but it did not work

URL url = new URL(mURL);
        final HttpURLConnection conn;
        conn = (HttpURLConnection) url.openConnection();
        conn.setConnectTimeout(10000);
        conn.connect();
        final BufferedInputStream in = new BufferedInputStream(conn.getInputStream());
        final File f = new File("tr.exe");
        if (f.exists()) {
            f.delete();
        }
        // open the output file and seek to the start location
        Thread t1 = new Thread() {
            public void run() {
                try {
                    RandomAccessFile raf = new RandomAccessFile(f, "rw");
                    raf.seek(0);
                    int numRead;
                    int mStartByte = 0;
                    byte data[] = new byte[conn.getContentLength() / 2];
                    while (((numRead = in.read(data, 0, 1024)) != -1) && (mStartByte < conn.getContentLength() / 2)) {
                        // write to buffer
                        raf.write(data, 0, numRead);
                        // increase the startByte for resume later
                        mStartByte += numRead;
                        // increase the downloaded size
                    }
                    raf.close();
                } catch (FileNotFoundException ex) {
                    Logger.getLogger(MyDownloader.class.getName()).log(Level.SEVERE, null, ex);
                } catch (IOException ex) {
                    Logger.getLogger(MyDownloader.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
        };
        Thread t2 = new Thread() {
            public void run() {
                try {
                    RandomAccessFile raf = new RandomAccessFile(f, "rw");
                    raf.seek(conn.getContentLengthLong() / 2);
                    int numRead;
                    int mStartByte = conn.getContentLength() / 2;
                    byte data[] = new byte[conn.getContentLength() / 2];
                    while (((numRead = in.read(data, 0, 1024)) != -1) && (mStartByte < conn.getContentLength())) {
                        // write to buffer
                        raf.write(data, 0, numRead);
                        // increase the startByte for resume later
                        mStartByte += numRead;
                        // increase the downloaded size
                    }
                    raf.close();
                } catch (FileNotFoundException ex) {
                    Logger.getLogger(MyDownloader.class.getName()).log(Level.SEVERE, null, ex);
                } catch (IOException ex) {
                    Logger.getLogger(MyDownloader.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
        };
        t1.start();
        t2.start();

score 0 · Answer 1 · answered Sep 24 '14 at 09:57

what is happening as i gues, is that you are writing to the same file at the same time, then yes the order change, and u may even lose some bytes in the way, now what you should do, is write the output of the first thread of a file called for example 'Thread1Output' and the second to 'Thread2Output' and at the end assemble the two files in one file, hope it works for u, and good luck, sorry, i cant provide code cause i have never create a downloader =D

score 0 · Accepted Answer · answered Sep 24 '14 at 10:08

Here's the first few problems I noticed...

First of all I would put your download/write code in a separate class that implements Runnable then execute it with an ExecutorService.
byte[] data should be the same size as the chunk you're reading which you've hardcoded as 1024.
int mStartByte = conn.getContentLength() / 2 what if the length is an odd number?
As you have 2 threads reading from same stream you are not guaranteeing that eg. 1st thread reads first half, 2nd thread reads second half. I suspect it won't work for files over 2048 bytes. Within each read iteration you need to know where in the stream you are starting so you can seek to the right place.

You can use a BlockingQueue if you want. What would you use it for? I would solve one issue at a time. — TedTrippin, Sep 24 '14 at 10:21
i have made an program that bing a song and find it as mp3 then download it than copy it my lg g2 automaticly xD — Karim, Sep 24 '14 at 10:34

Multi Thread java downloader downloading corrupted file?

2 Answers2