44

I'm searching in a HTML document using XPath from lxml in python. How can I get the path to a certain element? Here's the example from ruby nokogiri:

page.xpath('//text()').each do |textnode|
    path = textnode.path
    puts path
end

print for example '/html/body/div/div[1]/div[1]/p/text()[1]' and this is the string I want to get in python.

Fluffy
  • 27,504
  • 41
  • 151
  • 234

4 Answers4

68

Use getpath from ElementTree objects.

from lxml import etree

root = etree.fromstring('''
    <foo><bar>Data</bar><bar><baz>data</baz>
    <baz>data</baz></bar></foo>
    ''')

tree = etree.ElementTree(root)
for e in root.iter():
    print(tree.getpath(e))

Prints

/foo
/foo/bar[1]
/foo/bar[2]
/foo/bar[2]/baz[1]
/foo/bar[2]/baz[2]
Benjamin Loison
  • 3,782
  • 4
  • 16
  • 33
nosklo
  • 217,122
  • 57
  • 293
  • 297
  • 1
    I think it should be `for e in tree.iter():`, i.e. **tree**.iter. – Jabba Sep 17 '11 at 23:05
  • 2
    @Jabba And why do you think that? Have you tried the code I provided the way it is? It seems to work, no? Do you have a **reason** to think otherwise? – nosklo Sep 29 '11 at 15:11
  • It might have not existed when you wrote this originally and not that it actually matters, but you can also do `tree = root.getroottree()` to get an ElementTree object. – Kevin Vasko Apr 13 '17 at 18:36
  • 1
    Just for the sake of completion, when I use ```https://requests-html.kennethreitz.org/``` to pull a webpage, I do the following: ```webpage = session.get('URL HERE')``` then ```root = etree.HTML(webpage.text)``` then ```tree = etree.ElementTree(root)``` and then simply the for loop as stated in the answer. – JoHKa Nov 03 '21 at 02:19
20

See the Xpath and XSLT with lxml from the lxml documentation This gives the path of the element containg the text

An example would be

import cStringIO
from lxml import etree

f = cStringIO.StringIO('<foo><bar><x1>hello</x1><x1>world</x1></bar></foo>')
tree = lxml.etree.parse(f)
find_text = etree.XPath("//text()")

# and print out the required data
print [tree.getpath( text.getparent()) for text in find_text(tree)]

# answer I get is 
>>> ['/foo/bar/x1[1]', '/foo/bar/x1[2]']
mmmmmm
  • 32,227
  • 27
  • 88
  • 117
14

If all you have in your section of code is the element and you want the element's xpath do then element.getroottree().getpath(element) will do the job.

from lxml import etree

xml = '''
<test>
    <a/>
    <b>
       <i/>
       <ii/>
    </b>
</test>
'''
tree = etree.fromstring(xml)

for element in tree.iter():
    print element.getroottree().getpath(element)
shrewmouse
  • 5,338
  • 3
  • 38
  • 43
5
root = etree.parse(open('tmp.txt'))

for e in root.iter():
    print root.getpath(e)
hostingutilities.com
  • 8,894
  • 3
  • 41
  • 51