0

I am trying my first web scraping project in Jupyter notebook. When I try to find just div with the find_all function, It works well and returns the real length of containers.

all_page = doc.find_all('div')
len(all_page)

The answer is 262. But, when I try to add class, length returns 0

all_page = doc.find_all('div', class_="full-width artdeco-entity-lockup__title ember-view")
len(all_page)

Now, The answer of length is 0

I am attaching a picture of a site I am trying to get data from.Image

Can anyone please help me find the issue? I have read answers from here and tried to turn off JavaScript. But after doing that, I was unable to open the original website.

Khyati
  • 3
  • 3
  • JFYI: There's a difference between "scrapping" and "scraping". – Matthias Mar 14 '22 at 20:05
  • if the website is public, it's better if you provide a link to it. There are some pages that are dynamically generated by JavaScript, in this case it's necessary to use a library like selenium to automate user interaction to trigger that content creation. – Pepe N O Mar 14 '22 at 20:13
  • @Matthias corrected. Thanks . – Khyati Mar 16 '22 at 18:20

0 Answers0