1

i use selenium get web page and i send kenword get a new page. but how can i get the new web page ,and search the information that i need


browser = webdriver.Chrome()

test_url = 'https://www.baidu.com/'

browser.get(test_url)


in_put = browser.find_element_by_xpath('//*[@id="kw"]')
name = 'python'

in_put.send_keys(name.decode())
button = browser.find_element_by_xpath('//*[@id="su"]')
button.click()


page = browser.page_source

with open('baidu.html','wb') as f:
    f.write(page)

Forgive me for not using markdown. My question: i want to get the web page after search keyword, but i write browser.page_source , it is the Baidu's home page

Chen A.
  • 10,140
  • 3
  • 42
  • 61
Xiaoyi Lang
  • 21
  • 1
  • 3

1 Answers1

1

To get the current page URL using selenium on python use

browser.current_url

instead of

browser.page_source

Here is the changed code:

from selenium import webdriver

browser = webdriver.Chrome()
browser.get('https://www.baidu.com/')
in_put = browser.find_element_by_xpath('//*[@id="kw"]').send_keys('python')
button = browser.find_element_by_xpath('//*[@id="su"]').click()
page = browser.current_url
print(page)
with open('baidu.html', 'w') as f:
    f.write(page)

Output:

https://www.baidu.com/s?ie=utf-8&f=8&rsv_bp=0&rsv_idx=1&tn=baidu&wd=python&rsv_pq=c42791f700000914&rsv_t=d395wla6YQdEj168mrNDyE2%2FUMDzBUE3I%2FcwHbURsI%2FZ5TUaGFq1o83M2Qc&rqlang=cn&rsv_enter=0&rsv_sug3=6&inputT=610&rsv_sug4=611
Ali
  • 1,357
  • 2
  • 12
  • 18