0

I am trying to get some link for the website https://web.archive.org/web/*/https://cd.lianjia.com/, I want to get the link for each date,

import pandas as pd
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait

path = 'C:\Windows\chromedriver.exe'
driver = webdriver.Chrome(path)
driver.implicitly_wait(10)
driver.get('https://web.archive.org/web/*/https://cd.lianjia.com/ ')


element =driver.find_element_by_xpath('/html/body/div[4]/div[3]/div/div[2]/span[21]')
actions = ActionChains(driver)
actions.move_to_element(element).click().perform()

Because it has the data for several years, so I tried this to a specific year, 2016, now I am trying to get the href link for each date,

  day = driver.find_elements_by_css_selector("div.calendar-day")
for a in day:
    print(a.text)

But it only return number for days

      3
5
6
7
27
6
14
25
26
28
7
8
21
27
1
9
25
15
19
21
4
17
21
27
13
29
15
19
20
26
2
4

Then I tried this code to find the link

date = driver.find_elements_by_css_selector("a[href* = 'web']")
date = driver.find_elements_by_css_selector("a[href* = 'lianjia']")

but it cannot find the link, can someone help me with this, I have been stopped here for five days

1 Answers1

0

You approach is correct. but will need a some tweaks. You will need to target the elements directly and use get attribute method to extract the hrefs.

days = driver.find_elements_by_css_selector("div.calendar-day > a")
for day in days:
    print(day.get_attribute('href'))

output:

https://web.archive.org/web/20160103/https://cd.lianjia.com/
https://web.archive.org/web/20160105/https://cd.lianjia.com/
https://web.archive.org/web/20160106/https://cd.lianjia.com/
https://web.archive.org/web/20160107/https://cd.lianjia.com/
https://web.archive.org/web/20160127/https://cd.lianjia.com/
https://web.archive.org/web/20160306/https://cd.lianjia.com/
https://web.archive.org/web/20160314/https://cd.lianjia.com/
https://web.archive.org/web/20160325/https://cd.lianjia.com/
https://web.archive.org/web/20160326/https://cd.lianjia.com/
https://web.archive.org/web/20160328/https://cd.lianjia.com/
https://web.archive.org/web/20160407/https://cd.lianjia.com/
https://web.archive.org/web/20160408/https://cd.lianjia.com/
https://web.archive.org/web/20160421/https://cd.lianjia.com/
https://web.archive.org/web/20160427/https://cd.lianjia.com/
https://web.archive.org/web/20160501/https://cd.lianjia.com/
https://web.archive.org/web/20160509/https://cd.lianjia.com/
https://web.archive.org/web/20160525/https://cd.lianjia.com/
https://web.archive.org/web/20160615/https://cd.lianjia.com/
https://web.archive.org/web/20160619/https://cd.lianjia.com/
https://web.archive.org/web/20160621/https://cd.lianjia.com/
https://web.archive.org/web/20160704/https://cd.lianjia.com/
https://web.archive.org/web/20160717/https://cd.lianjia.com/
https://web.archive.org/web/20160721/https://cd.lianjia.com/
https://web.archive.org/web/20160927/https://cd.lianjia.com/
https://web.archive.org/web/20161013/https://cd.lianjia.com/
https://web.archive.org/web/20161029/https://cd.lianjia.com/
https://web.archive.org/web/20161115/https://cd.lianjia.com/
https://web.archive.org/web/20161119/https://cd.lianjia.com/
https://web.archive.org/web/20161120/https://cd.lianjia.com/
https://web.archive.org/web/20161126/https://cd.lianjia.com/
https://web.archive.org/web/20161202/https://cd.lianjia.com/
https://web.archive.org/web/20161204/https://cd.lianjia.com/
Sureshmani Kalirajan
  • 1,938
  • 2
  • 9
  • 18