Less painful way to parse a RSS-Feed with lxml?

Question

I need to display RSS-feeds with Python, Atom for the most part. Coming from PHP, where I could get values pretty fast with $entry->link i find lxml to be much more precise, faster, albeit complicated. After hours of probing I got this working with the arstechnica-feed:

def GetRSSFeed(url):
    out = []
    feed = urllib.urlopen(url)
    feed = etree.parse(feed)
    feed = feed.getroot()
    for element in feed.iterfind(".//item"):
        meta = element.getchildren()
        title = meta[0].text
        link = meta[1].text
        for subel in element.iterfind(".//description"):
            desc = subel.text
            entry = [title,link,desc]
            out.append(entry)
    return out

Could this be done any easier? How can I access tags directly? Feedparser gets the job done with one line of code! Why?

Why are you using lxml instead of feedparser, then? – bgporter Jun 22 '12 at 14:24 — bgporter, Jun 22 '12 at 14:24

score 9 · Answer 1 · answered Jun 22 '12 at 14:37

9

Look at the feedparser library. It gives you a nicely formatted RSS object.

> import feedparser
> feed = feedparser.parse('http://feeds.marketwatch.com/marketwatch/marketpulse/')
> print feed.keys()
['feed',
 'status',
 'updated',
 'updated_parsed',
 'encoding',
 'bozo',
 'headers',
 'etag',
 'href',
 'version',
 'entries',
 'namespaces']

>  len(feed.entries)
    30

answered Jun 22 '12 at 14:37

guyrt

927
7
12

thanks for the answer. i mentioned feedparser in the op. i tested it against lxml, which came out much faster. all i wanna do now is select children by their tag name. like rss.item.description.text. impossible? – reinhardt Jun 22 '12 at 14:54
1

Is this more in line with what you want? (Find all descriptions that are children of items) feed.findall('.//item//description') – guyrt Jun 22 '12 at 15:16
Not an answer to what is being asked . can you do this hard way using lxml – Harshit May 15 '13 at 13:46

rubayeet · Answer 2 · 2014-04-20T04:12:28.233

3

You can try speedparser, an implementation of Universal Feed Parser with lxml. Still in beta though.

edited Apr 20 '14 at 04:12

answered Jun 19 '13 at 06:32

rubayeet

9,269
8
46
55

Less painful way to parse a RSS-Feed with lxml?

2 Answers2