java.io.eofexception when downloading files from remote server to android phone

Question

My objective is to download 200 .jpg files that are in a remote server to my android phone (running jellybeans). In order to do that I'm running below method in a loop with different file names assigned to the filename parameter. It runs well untill I download 70 files. after that I get an java.io.eofexception. Can you please help.

protected void doDownloadPathwayImageFiles(final String fileName, final int totalNumberOfFiles, final int downloadedFiles) throws Exception {


                File root = android.os.Environment.getExternalStorageDirectory();
                File dir = new File(root.getAbsolutePath() + "/" + UtilConstants.PATHWAY_IMAGE_FOLDER + "/");
                if (dir.exists() == false) {
                    dir.mkdirs();
                }

                try {
                    URL url = new URL(UtilConstants.PATHWAY_FILE_LOCATION + fileName + UtilConstants.IMAGE_FILE_EXTENSION);
                    Log.i("FILE_NAME", "File name is " + fileName);
                    Log.i("FILE_URLLINK", "File URL is " + url);

                    HttpURLConnection conn = (HttpURLConnection) url.openConnection();
                    //conn.setReadTimeout(10000 /* milliseconds */);
                    //conn.setConnectTimeout(15000 /* milliseconds */);
                    conn.setRequestMethod("GET");
                    conn.setDoInput(true);
                    conn.setRequestProperty("Connection", "close");
                    // Starts the query
                    conn.connect();
                    int response = conn.getResponseCode();
                    Log.i("NETWORK_RESPONSE", "The response is___: " + response);

                    // download the file
                    InputStream urlStream = conn.getInputStream();
                    InputStream input = new BufferedInputStream(urlStream);

                    OutputStream output = new FileOutputStream(dir + "/" + fileName + UtilConstants.IMAGE_FILE_EXTENSION);

                    //write the file to local directory
                    byte data[] = new byte[1024];
                    long total = 0;
                    int count;
                    while ((count = input.read(data)) != -1) {
                        total += count;
                        output.write(data, 0, count);
                    }

                    output.flush();
                    output.close();
                    input.close();
                    urlStream.close();
                    conn.disconnect();



                    Log.i("Files", "Downloaded " + downloadedFiles);
                    Log.i("Files", "Total " + totalNumberOfFiles);

                    double progressCount = ((double) downloadedFiles / (double) totalNumberOfFiles) * 100;

                    Log.i("percentage", "Progress ++++++++++ " + progressCount);

                    progressBar.setProgress((int) Math.round(progressCount));

                }catch (MalformedURLException e) {
                    throw new MalformedURLException("Error downloading pathway overview images :" + e.getMessage());
                }  catch (IOException e){
                    throw new IOException("Error downloading pathway overview images :" + e.getMessage());
                }  catch (Exception e) {
                    throw new Exception("Error downloading pathway overview images :" + e.getMessage());
                }

        }

Is it a specific file? Or any 70 files? If it's a specific file, what happens when you try to load that file in a browser? — Paul Coghill, Jun 03 '15 at 15:22
Paul, It's not a specific file (this file loads fine in the browser). it happens after reading 70 files. — Asanga Dewaguru, Jun 03 '15 at 16:22
Sorry Paul, Just figured out that, this error is triggered only for the 8 particular files out of the total of 200. have to find what's wrong with these 8 files — Asanga Dewaguru, Jun 04 '15 at 02:39

score 1 · Answer 1 · answered Jun 03 '15 at 13:55

1

Maybe that file is corrupted somehow ?

Have you tried to make that operation again after you recevied that error ?

For example after you throw Exception:

 catch (Exception e) {
                throw new Exception("Error downloading pathway overview images :" + e.getMessage());
            }

catch it and invoke that method protected void doDownloadPathwayImageFiles(final String fileName, final int totalNumberOfFiles, final int downloadedFiles) throws Exception again in your code(ie: for the 3rd time just ignore file and try download next).

answered Jun 03 '15 at 13:55

Paweł Głowacz

2,926
3
18
25

Pawel, I've tried the retry and ignore strategy that you've explained and 8 files out 200 are getting failed to download. Thanks for the heads up. I still wonder why these 8 files are getting EOF exception. These files are getting opened well in the browser, therefore seems not corrupted. trying to figure it out now – Asanga Dewaguru Jun 04 '15 at 02:26
Could you invoke getContentLength() on HttpUrlConnection and see what it returns ? If it returns 0 length that could be an issue here. – Paweł Głowacz Jun 04 '15 at 18:16
exactly it returns 0 for the length. and response code is also not 200 as it is in successful scenarios. Does this mean there is a problem with the image file itself? – Asanga Dewaguru Jun 05 '15 at 10:01
In that case please invoke method `getHeaderField("Transfer-Encoding")` on `HttpUrlConnection` and check what it returns. My guess is: "**chunked**". If yes please refer to this: http://docs.oracle.com/javase/7/docs/api/java/net/HttpURLConnection.html#setChunkedStreamingMode(int) Try to use `setChunkedStreamingMode(-1)` or `setChunkedStreamingMode(0)` before `connect`and see result. – Paweł Głowacz Jun 05 '15 at 11:39
I just checked the getHeaderField("Transfer-Encoding"). It always returns "null" regardless of the fact whether it's a successfully downloaded image or not. – Asanga Dewaguru Jun 08 '15 at 01:10
I can't believe. I just found out that I'm getting these error for below file names (txt and Jpg) Common pattern for these files are having " H" (notice the space before the character H) Once I rename these files by replacing " H" with a random letter like " K". It works very well. Files are:- 1.)"Raised Hemidiaphragm on CXR" 2.)"Paediatric, Hydronephrosis (Antenatal)" 3.)"Paediatric, Head Trauma" 4.)"Paediatric, Headache" 5.)"Paediatric, Headache (Recurrent)" 6.)"Paediatric, Hip Developmental Dysplasia" 7.)"Paediatric, Hip Pain" 8.)"Paediatric, Hydronephrosis (Antenatal)" – Asanga Dewaguru Jun 09 '15 at 10:53

Paul Coghill · Answer 2 · 2015-06-05T10:20:30.440

1

This is probably because you are constantly reading 1024 bytes from the file, which may be more than there is left to read.

You would want to check the Content Length of the file you are downloading and only read what you need to.

I would also check that the response code is 200 (OK). And the Content Length header of the response returned by the server:

long contentLength = Long.parseLong(connection.getHeaderField("Content-Length"));

Do not attempt to proceed to read the file if this is < 1 as it won't be valid. The Content-Length is the literal size of the file being returned, so you will get an End of File exception if you try to read this, as the there is no data to read in the response.

edited Jun 05 '15 at 10:20

answered Jun 03 '15 at 15:39

Paul Coghill

667
6
27

Paul, EOF exception triggers when it calls conn.getInputStream(); that is before it reads the bytes. – Asanga Dewaguru Jun 04 '15 at 01:11
Added a couple more suggestions. – Paul Coghill Jun 04 '15 at 07:42
Correct, Paul. content length is 0. I just checked it. does it mean that there is a problem with the image file it self? – Asanga Dewaguru Jun 05 '15 at 10:02
Yes, that means that no data will be returned as the file is empty or corrupt. If the Http Status Code is 200, then it's probably a corrupt file, if it's something like 404 (Not found) then it could point to you the actual reason for the request not working. – Paul Coghill Jun 05 '15 at 10:19

score 0 · Accepted Answer · answered Jun 09 '15 at 13:46

Finally managed to find out the exact reason. I was connecting to a nginx server (testing server) in order to get the corresponding files downloaded. There is this reported issue "Inconsistent behavior on uri's with unencoded spaces followed by H" (http://trac.nginx.org/nginx/ticket/196) in nginx. Therefore nginx returns bad request error back to the client. Which eventually throws an eof exception in connection.openStram().

Since the production version of the software that I'm working on doesn't required to connect to an nginx instance. I just changed the server, not using nginx anymore. But still I had to do replace the spaces in the URL with '%20' using String.replace() method.

java.io.eofexception when downloading files from remote server to android phone

3 Answers3