1

I'm trying to extract data from a table(1), which has a couple filter options. I'm using BeautifulSoup and got to this page with Requests. An extract of code: 

from bs4 import BeautifulSoup


tt = Contact_page.content # webpage with table
soup = BeautifulSoup(tt)
R_tables = soup.find('div', {'class': 'responsive-table'})

Using find_all("tr") and find_all("th") results in empty sets. Using R_tables.findChildren only goes down to "formrow" who then has no children. From formrow to my tr/th tags, I can't access it through BS4.

R_tables results in table 3. The XPath for this file is 

"//*[@id="kronos_body"]/div[3]/div[2]/div[3]/script/text()

How can I get each row information for my data? soup.find("r") and soup.find("f") also result in empty sets.

Pardon me in advance if this post is sloppy, this is my first. I'll link what my most similar thread is in a comment, I can't link more than 2 times.

EDIT 1 : Apparently BS doesn't recognize any javascript apart from variables (correct me if I'm wrong, I'm still still relatively new). Are there any other modules that can help me out? I was proposed Ghost and Selenium, but I won't be using Selenium.

Martin Gergov
  • 1,556
  • 4
  • 20
  • 29
CTVDevine
  • 43
  • 8
  • Most similar thread http://stackoverflow.com/questions/28635818/scraping-data-from-a-dynamic-web-database-with-python?rq=1 – CTVDevine May 14 '15 at 14:24
  • not a bad first post, remember you can always clean your post up using the edit button. Welcome to stack overflow, recommended reading: http://stackoverflow.com/help/how-to-ask I share this link with all first time posters that I come across. – Dan Beaulieu May 14 '15 at 14:27
  • 1
    I've got some editing to do then. Thanks – CTVDevine May 14 '15 at 14:35

0 Answers0