Having trouble reading in content of url using InputStream

Question

So I run the code below and it prints "!DOCTYPE html". How do I get the content of the url, like the html for instance?

public static void main(String[] args) throws IOException {
        URL u = new URL("https://www.whitehouse.gov/");
        InputStream ins = u.openStream();
        InputStreamReader isr = new InputStreamReader(ins);
        BufferedReader websiteText = new BufferedReader(isr);
        System.out.println(websiteText.readLine());

    }

According to java doc https://docs.oracle.com/javase/tutorial/networking/urls/readingURL.html: "When you run the program, you should see, scrolling by in your command window, the HTML commands and textual content from the HTML file located at ".... Why am I not getting that?

The code you posted isn't the same as the code from the tutorial you cited, so there is no reason why it should behave the same. — user207421, Oct 15 '19 at 11:27

score 1 · Answer 1 · answered Oct 15 '19 at 10:18

1

You are only reading one line of the text. Try this and you will see that you get two lines:

System.out.println(websiteText.readLine());
System.out.println(websiteText.readLine());

Try reading it in a loop to get all the text.

answered Oct 15 '19 at 10:18

Torkel Velure

1,217
10
17

score 1 · Accepted Answer · answered Oct 15 '19 at 10:23

In your program, your did not put while loop.

   URL u = new URL("https://www.whitehouse.gov/");
    InputStream ins = u.openStream();
    InputStreamReader isr = new InputStreamReader(ins);
    BufferedReader websiteText = new BufferedReader(isr);
    String inputLine;
    while ((inputLine = websiteText.readLine()) != null){
        System.out.println(inputLine);
   }

  websiteText.close();

score 0 · Answer 3 · answered Oct 15 '19 at 10:24

BufferedReader has a method called #lines() since Java 8. The return type of #lines() is Stream. To read an entire site you could do something like that:

String htmlText = websiteText.lines()
  .reduce("", (text, nextLine) -> text + "\n" + nextLine)
  .orElse(null);

Having trouble reading in content of url using InputStream

3 Answers3