Reading a web page in Java IOException Premature EOF

Question

I am frequently getting a 'Premature EOF' Exception when reading a web page.

The following is the StackTrace

java.io.IOException: Premature EOF
    at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:556)
    at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:600)
    at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:687)
    at java.io.FilterInputStream.read(FilterInputStream.java:133)
    at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2968)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
    at java.io.InputStreamReader.read(InputStreamReader.java:184)
    at java.io.BufferedReader.fill(BufferedReader.java:154)
    at java.io.BufferedReader.readLine(BufferedReader.java:317)
    at java.io.BufferedReader.readLine(BufferedReader.java:382)
    at Utilities.getPage(Utilities.java:24)  while ((line = rd.readLine()) != null) {
    at TalkPage.<init>(TalkPage.java:15)
    at Updater.run(Updater.java:65)

Following is the getPage() method

public static String getPage(String urlString) throws Exception {
    URL url = new URL(urlString);
    URLConnection conn = url.openConnection();
    BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
    StringBuffer sb = new StringBuffer();
    String line;
    while ((line = rd.readLine()) != null) {  // LINE 24
        sb.append(line);
    }
    return sb.toString();
}

What is a permature EOFException and why is it occuring in this particular case and how can it be avoided?

Some other information: The size of the page being read is around 20 KB and I'm reading many such pages in my program ( around 20 000 )

dan · Accepted Answer · 2014-07-18T07:16:02.760

8

This may be because you are reading the content line by line and for the last line the file may be missing a return, to signal the end of line. Replace your while with this:

int BUFFER_SIZE=1024;
char[] buffer = new char[BUFFER_SIZE]; // or some other size, 
int charsRead = 0;
while ( (charsRead  = rd.read(buffer, 0, BUFFER_SIZE)) != -1) {
  sb.append(buffer, 0, charsRead);
}

edited Jul 18 '14 at 07:16

answered Nov 03 '12 at 14:44

dan

13,132
3
38
49

and for the last line the file may be missing a return ::: When I am reading the same web page a second time, I am not getting any error. So, I think that might not be the problem. And, can you answer the other part of the question also : What is a permature EOFException – Ranjith Nov 03 '12 at 15:48
1

The "permature EOFException" is happening when you are expecting something, like here a line termination, but there is none in the stream. – dan Nov 03 '12 at 16:13
1

@Ranjith-SR2GF Have you tried my fix? It should work each time. The issue for you it may be that the server is not pushing that last end line character. – dan Nov 03 '12 at 16:14
For now, I have finished downloading the required web pages. I shall try it the next time I need to do a similar job. Thanks for the answer. – Ranjith Nov 03 '12 at 17:30
@brgr Not sure why your edit was rejected, it was a good edit, thanks. – dan Jul 18 '14 at 07:18
@dan I have very similar code and this error still occurs. – mjs Jan 04 '15 at 07:48
@momo You should submit a new question with your code. This way we will be able to discuss what is different. – dan Jan 04 '15 at 08:39
2

why this will not cause Premature EOF , Premature EOF comes when no signal is returned for an end of stream and still the bytes/characters stops coming , how calling rd.read() will help here , Might be solution is correct but if you can give an explanation, will be much helpful. – Adon Smith Jun 18 '15 at 13:58

score 5 · Answer 2 · answered May 23 '14 at 21:00

This could be because the server is closing the connection. I have experienced the exact same issue when I had a piece of code which opened a connection, did some other processing, and only then tried to download the contents of the input stream - by the time it to the stream after spending a few seconds on other processing, the server had apparently closed the connection, resulting in IOException: Premature EOF. The solution was to be careful to always immediately handle the contents of the stream - otherwise, you are leaving an HTTP connection open and idle, and eventually the server on the other end of the line will hang up on you.

This could also be because the server is only sending headers and no body as response. — cachius, Nov 18 '20 at 13:36

score 0 · Answer 3 · edited Nov 03 '12 at 15:47

0

You can use APACHE commons io FileUtils.copyURLToFile method

http://commons.apache.org/io/api-release/org/apache/commons/io/FileUtils.html#copyURLToFile%28java.net.URL,%20java.io.File,%20int,%20int%29

edited Nov 03 '12 at 15:47

Aitul

2,982
2
24
52

answered Nov 03 '12 at 15:14

Software companies in perth

9
1

I want to get the contents of the URL into a String and I do not want to write it to a file and read it back again. – Ranjith Nov 03 '12 at 15:49
It would take another LOC to read it from file to String... http://commons.apache.org/io/api-release/org/apache/commons/io/FileUtils.html#readFileToString%28java.io.File%29 – Software companies in perth Nov 17 '12 at 05:12
Why do you think this is better? – Wilfred Springer Nov 28 '14 at 11:23
@Aitul Above mentioned links are not working please update. – Chaminda Bandara Nov 15 '19 at 12:04
`org.apache.commons.io.IOUtils.toString(InputStream, Charset)` can be used to read from a URL into a String without any temporary file. – Paul Podgorsek Jul 07 '23 at 09:32

score 0 · Answer 4 · answered Mar 06 '15 at 16:54

0

You can also try to set the buffer size to 1. This slightly helps and if you implement a try logic around it, then it should do the trick.

answered Mar 06 '15 at 16:54

Ducaz035

3,054
2
25
45

André Luiz Miranda · Answer 5 · 2015-04-01T06:08:17.117

        StringBuilder sb = new StringBuilder();

        try{                
            URL url = new URL(address);

            InputStream is = url.openStream();

            InputStreamReader isr = new InputStreamReader(is);

            BufferedReader in = new BufferedReader(isr);

            String str;

            while((str = in.readLine()) != null){

                sb.append(str);

                sb.append("\n");
            }

            in.close();
            isr.close();
            is.close();

            return sb.toString();

         }catch(Exception e){

             //OMG....
         }

Reading a web page in Java IOException Premature EOF

5 Answers5

Linked