Python Selenium - How to loop to the last element in a site

Question

I have created a python selenium script that should navigate through a website and collect people profiles (https://www.shearman.com/people). The program won't loop through the pages to collect the links. I have used this which doesn't work;

 try:
     # this is navigate to next page
     driver.find_element_by_xpath('//div[@id="searchResultsSection"]/ul/li[12]').click()
     time.sleep(1)
 except NoSuchElementException:
     break

The syntax behind the next button can be seen below;

<a href="" onclick="PageRequest('2', event)" class="xh-highlight">&gt;</a>

Does anybody no how to write the code to click the next button?

Answer 1

If you look at the HTML of the element with image as >, it is the <a> tag within the last <li> tag of the <ul> tag. So to invoke click() on it you can use the following code block :

driver.find_element_by_xpath("//ul[@class='results-pagination']/li[last()]/a").click()

Answer 2

you can do it yourself :

• ctrl+shift+i
• right click on the next button -> inspect
• right click on the next button code -> copy -> copy selector

then you store the button in a WebElement nextButton then

nextButton.click();

Answer 3

You can try to use below line to click Next button

driver.find_element_by_link_text(">").click()