0

I'm using the find_element_by_partial_link_text selector to find the "next" button, so that I can click it and continue crawling.

However, the issue I'm having is that sometimes the word "next" is also in other random links on the page, which breaks the script.

I've tried addressing this with the code below, which obviously doesn't work.

pagination_ul = ff.find_element_by_xpath('//*[contains(@id, "pagination-both-")]')
next_button = pagination_ul.find_element_by_partial_link_text('Next')

So my question is...

How can I retrieve and compare the parent element of a find_element_by_partial_link_text element?

In other words, how can I make sure that the parent of the find_element_by_partial_link_text is, say, a li element?

oldboy
  • 5,729
  • 6
  • 38
  • 86
  • can you add the relevant HTML code – NarendraR Sep 19 '18 at 04:33
  • @NarendraR here's a [link](https://www.amazon.ca/gp/goldbox/ref=gbps_ftr_s-3_4bc8_dct_10-?gb_f_c2xvdC0z=sortOrder:BY_SCORE,discountRanges:10-25%252C25-50%252C50-70%252C70-&pf_rd_p=f5836aee-0969-4c39-9720-4f0cacf64bc8&pf_rd_s=slot-3&pf_rd_t=701&pf_rd_i=gb_main&pf_rd_m=A3DWYIK6Y9EEQB&pf_rd_r=CQ7KBNXT36G95190QJB1&ie=UTF8) to the website i'm crawling – oldboy Sep 19 '18 at 04:56
  • You may have identifiy the correct parent node – Pradeep hebbar Sep 19 '18 at 05:06

2 Answers2

2

Try using below XPath:

driver.find_element_by_xpath('//div[starts-with(@id, "pagination-") and not(contains(@class, "hidden"))]//li[@class="a-last"]/a[text()="Next"]').click()

It looks more complicated then search by text content, but should work in all cases

Note that you might need to Wait for element to appear in DOM/to be clickable

Andersson
  • 51,635
  • 17
  • 77
  • 129
0

If you want to find parent element of an element, then use the below approach:

For instance, let us take https://www.google.com webpage. The xpath for the mike icon is //div[@id='gs_st0']. To find the parent of that element, which is the search box, use 'child' axes as below.

//*[child::div[@id='gs_st0']]

The above xpath will fetch you the parent element of the child.

In your case, if your child element's xpath is: //a[text()='Next'] So to find the parent element of the 'Next' link, use below xpath:

parentElement = driver.find_element_by_xpath("//*[child::a[text()='Next']]")

Kireeti Annamaraj
  • 1,037
  • 1
  • 8
  • 12
  • i can't use xpath. i have to use `find_element_by_partial_link_text` for [this reason.](https://stackoverflow.com/questions/52356838/issue-crawling-amazon-element-cannot-be-scrolled-into-view) – oldboy Sep 19 '18 at 04:08