How to extract the name of the authors using an XPath expression

Question

Given below is the snippet of HTML:

<div class="a-row a-spacing-none">
    <span class="a-size-small a-color-secondary">by 
    </span>
    <span class="a-size-small a-color-secondary">
        <a class="a-link-normal a-text-normal" href="/Lowell-Fryman/e/B01M3MNJTE/ref=sr_ntt_srch_lnk_1?qid=1550228622&amp;sr=1-1">
        Lowell Fryman
        </a> 
        and 
    </span>
    <span class="a-size-small a-color-secondary">
        <a class="a-link-normal a-text-normal" href="/Gregory-Lampshire/e/B01N7ZWT5Y/ref=sr_ntt_srch_lnk_1?qid=1550228622&amp;sr=1-1">
        Gregory Lampshire
        </a>
    </span>
</div>

I'm trying to obtain the name of all authors. This is whatever test follows the word by. I came up with the following XPath but it doesn't seem to fetch all the authors.

My XPath expression:

//div//span[text()=\"by \"]//following::span[1]//a

Can someone please tell me how to obtain the name of all the authors while somehow managing to skip any element whose text is "and"?

I'm using Selenium's find_element_by_xpath if it helps.

Sers · Answer 1 · 2019-02-15T12:54:44.677

1

Here xpath you can use to get authors:

//div[./span[normalize-space(.='by')]]//a

or

//div[./span[contains(.,'by')]]//a

Your xpath should be like this:

//span[normalize-space(.='by')]/following-sibling::span//a

edited Feb 15 '19 at 12:54

answered Feb 15 '19 at 11:24

Sers

12,047
2
12
31

score 1 · Accepted Answer · answered Feb 15 '19 at 11:24

1

Try the below code.This should work.It will fetch all author.

elements=driver.find_elements_by_xpath("//a[@class='a-link-normal a-text-normal']")

for element in elements:
    print(element.text)

Please let me know if this work.

answered Feb 15 '19 at 11:24

KunduK

32,888
5
17
41

score 1 · Answer 3 · answered Feb 15 '19 at 12:57

1

you can first get text of all span elements in a list and then slice it from "by" text value

elements = [_.text() for _ in driver.find_elements_by_css_selector('div.span')]
print elements[elements.index('by'):]

answered Feb 15 '19 at 12:57

Jagan Nagare

36
4

score 0 · Answer 4 · answered Feb 15 '19 at 14:42

To print the name of all the authors which are followed by the word by you can use either of the following solutions:

Using innerHTML:

print([author.get_attribute("innerHTML") for author in driver.find_elements_by_xpath("//span[contains(., 'by')]//following::span/a[@class='a-link-normal a-text-normal'][@href]")])

Using text:

print([author.text for author in driver.find_elements_by_xpath("//span[contains(., 'by')]//following::span/a[@class='a-link-normal a-text-normal'][@href]")])

How to extract the name of the authors using an XPath expression

4 Answers4