6

I am trying to scrape an airbnb listing. I cant figure out a way to get the full list of amenities other than clicking on "more". I am using selenium to simulate the click, but it does nt seem to work.

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

url = 'https://www.airbnb.com/rooms/4660676'
driver = webdriver.Firefox()
driver.get(url)
elem = driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]')
actions.click(elem).perform()
mitghi
  • 889
  • 7
  • 20
Borat14
  • 75
  • 1
  • 1
  • 4

2 Answers2

9

The XPath itself is correct, but you have not defined the actions:

from selenium.webdriver.common.action_chains import ActionChains

elem = driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]')

actions = ActionChains(driver)
actions.click(elem).perform()

Note that you can simply use the WebElement's click() method instead:

elem = driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]')
elem.click()

Works for me.


If you are getting NoSuchElementException error, you might need to wait for the link to be clickable via WebDriverWait and element_to_be_clickable.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

element = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, '//a[@class="expandable-trigger-more"]'))
)
element.click()
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
  • @Borat14 works for me, you are doing smth more than just clicking more button probably. Post the complete code you have so far. Thanks. – alecxe Jan 24 '16 at 05:30
  • Thanks @alecxe, on the first methods i get this error: WebDriverException: Message: Element is not clickable at point (661.5, 10.316665649414062). Other element would receive the click: on the last two methods i get "string indices must be integers" – Borat14 Jan 24 '16 at 05:49
  • This is the code `from selenium import webdriver from selenium.webdriver.common.action_chains import ActionChains url = 'https://www.airbnb.com/rooms/4660676' driver = webdriver.Firefox() driver.get(url) elem = driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]') actions = ActionChains(driver) actions.click(elem).perform()` – Borat14 Jan 24 '16 at 05:50
  • @Borat14 the "string indexes must be integers" is because of the selenium 2.49 bug, downgrade to 2.48: `pip install selenium==2.48` (http://stackoverflow.com/questions/34969006/python-selenium-click-and-typeerror-string-indices-must-be-integers). – alecxe Jan 24 '16 at 05:51
1

A very simple way of achieving this is given below

driver.find_element_by_xpath('//a[@class="expandable-trigger-more"]').click()

It works for me hope will work for you as well.

Mohsin Ashraf
  • 972
  • 12
  • 18