1

I am trying to read an html page in my java code using Jsoup library. This is the link to the page: http://www.alkhaleej.ae/

The part in the page that I am interested in is the horizontal menu bar at the top of the page (which has the news categories). When I right click on that menu bar and choose inspect element, the html elements of interest are visible to me under the tag <div id="MainMenuCenter">. However, when I run my code, it turns out this tag is actually empty, and all the children of this tag get invisible. I also tried to view the complete document using "view page source" on the webpage. I surprisingly found this element empty (no children) as below.

<div id="MainMenuCenter">

</div>

Therefore, I am not able to access the information I need in my code. What is really going on? Did the developers hide the children of this element on purpose? Can you suggest a way to make the children visible to my code? Thank you.

Traveling Salesman
  • 2,209
  • 11
  • 46
  • 83

1 Answers1

1

You can retrieve the data by looking at the network traffic on

Inspect element -> Network

Check the traffic one by one or use the find tools. If you find the match data, you can re-obtain it by visiting the url who serve the data..

Maybe like: http://example.com/serve.php?category=car&page=1

StefansArya
  • 2,802
  • 3
  • 24
  • 25
  • I hope if you can elaborate. I have checked the network section but the URLs there looks nothing like the link you provided. Also, how am I supposed to figure out which one is the one I am looking for? Finally, would it give a URL that has the div subelements I am looking for or what kind of data the URL would have? – Traveling Salesman Feb 19 '17 at 21:23
  • Have you check at the url's response in the developer tools? – StefansArya Feb 20 '17 at 09:38
  • The other way to do is by checking if the javascript contain any words for `MainMenuCenter` – StefansArya Feb 20 '17 at 09:39