Python/XML RSS Feedparser - Blank Fields

Question

I'm trying to parse the NOAA Weather RSS Feed. I've got my script pulling the entries just fine, but some of the fields are blank and I can't figure out why. It seems to be a problem with more complex data types in the feed. Here is the code:

url = 'http://alerts.weather.gov/cap/us.php?x=1'
feed = feedparser.parse(url)
print 'title: ', feed.entries[0].title
print 'cap_geocode: ', feed.entries[0].cap_geocode
print 'cap_event: ', feed.entries[0].cap_event

Here are the results from printing an entry:

title:  Flood Watch issued June 18 at 5:00AM AKDT until June 21 at 4:00PM...
cap_geocode:  
cap_event:  Flood Watch

I can't figure out why cap_geocode is blank. Any idea? Here is the xml file with just one element. The feed actually pulls back 100s or entries, so I reduced it to just a small sample.

<feed
xmlns = 'http://www.w3.org/2005/Atom'
xmlns:cap = 'urn:oasis:names:tc:emergency:cap:1.1'
xmlns:ha = 'http://www.alerting.net/namespace/index_1.0'
>

<id>http://alerts.weather.gov/cap/us.atom</id>
<logo>http://alerts.weather.gov/images/xml_logo.gif</logo>
<generator>NWS CAP Server</generator>
<updated>2014-06-18T08:22:00-06:00</updated>
<author>
<name>w-nws.webmaster@noaa.gov</name>
</author>
<title>Current Watches, Warnings and Advisories for the United States Issued by the National Weather Service</title>
<link href='http://alerts.weather.gov/cap/us.atom'/>
    <entry>
    <id>http://alerts.weather.gov/cap/wwacapget.php?x=WY12515A62F544.WinterWeatherAdvisory.12515A71D320WY.RIWWSWRIW.8dcf1ff2519d541d79867824b0480d63</id>
    <updated>2014-06-18T02:57:00-06:00</updated>
    <published>2014-06-18T02:57:00-06:00</published>
    <author>
    <name>w-nws.webmaster@noaa.gov</name>
    </author>
    <title>Winter Weather Advisory issued June 18 at 2:57AM MDT until June 19 at 12:00AM MDT by NWS</title>
    <link href='http://alerts.weather.gov/cap/wwacapget.php?x=WY12515A62F544.WinterWeatherAdvisory.12515A71D320WY.RIWWSWRIW.8dcf1ff2519d541d79867824b0480d63'/>
    <summary>...WINTER WEATHER CONTINUES TO IMPACT THE WESTERN AND CENTRAL MOUNTAINS THROUGH MIDNIGHT.</summary>
    <cap:event>Winter Weather Advisory</cap:event>
    <cap:effective>2014-06-18T02:57:00-06:00</cap:effective>
    <cap:expires>2014-06-19T00:00:00-06:00</cap:expires>
    <cap:status>Actual</cap:status>
    <cap:msgType>Alert</cap:msgType>
    <cap:category>Met</cap:category>
    <cap:urgency>Expected</cap:urgency>
    <cap:severity>Minor</cap:severity>
    <cap:certainty>Likely</cap:certainty>
    <cap:areaDesc>Absaroka Mountains; Teton and Gros Ventre Mountains</cap:areaDesc>
    <cap:polygon></cap:polygon>
    <cap:geocode>
    <valueName>FIPS6</valueName>
    <value>056013 056017 056023 056029 056035 056039</value>
    <valueName>UGC</valueName>
    <value>WYZ002 WYZ012</value>
    </cap:geocode>
    <cap:parameter>
    <valueName>VTEC</valueName>
    <value>/O.CON.KRIW.WW.Y.0031.000000T0000Z-140619T0600Z/</value>
    </cap:parameter>
    </entry>    
</feed>

Any help would be appreciated. Thanks

*Update I have verified that cap_geocode is a blank unicode string of length 0, not an object of some sort.

score 1 · Accepted Answer · edited May 23 '17 at 12:05

It seems you're supposed to monkeypatch feedparser ...unfortunately the xml parser does not seem to parse attributes for cap_geocode.

I'm not an xml expert but ElementTree also does not seem to parse attributes for cap_geocode.

At some point I stopped fighting it and just wrote:

from bs4 import BeautifulSoup
import requests

url = 'http://alerts.weather.gov/cap/us.php?x=1'

r = requests.get(url)
soup = BeautifulSoup(r.text, 'xml')
entries = soup.find_all('entry')

geocode = {}
k = None
for tag in entries[0].geocode.children:
    if tag.name == 'valueName':
        k = tag.string
    elif tag.name == 'value':
        geocode[k] = tag.string

print 'title: ', entries[0].title.string
print 'cap_geocode: ', str(geocode)
print 'cap_event: ', entries[0].event.string

Wow. It took me forever to get the script working cause of some dependencies, but it does work! — Hoopdady, Jun 25 '14 at 13:00

score 0 · Answer 2 · answered Jun 18 '14 at 18:59

So, your cap_geocode variable actually points to a node with children in the RSS feed:

<cap:geocode>
   <valueName>FIPS6</valueName>
   <value>056013 056017 056023 056029 056035 056039</value>
   <valueName>UGC</valueName>
   <value>WYZ002 WYZ012</value>
</cap:geocode>

So, you'll probably have to do something a bit more complex like this feed.entries[0].cap_geocode[0].valueName[0] to access FIPS6, feed.entries[0].cap_geocode[0].valueName[1] to access UGC, feed.entries[0].cap_geocode[0].value[0] to access 056013 056017 056023 056029 056035 056039, and feed.entries[0].cap_geocode[0].value[1] to access WYZ002 WYZ012.

Actually feed.entries[0].cap_geocode is a unicode string of length 0. SO that will not work. The issue seems to be that its not grabbing the data. You can see for yourself. You'll have to install `feedparser` then import it but the script I posted works. — Hoopdady, Jun 18 '14 at 19:46

Python/XML RSS Feedparser - Blank Fields

2 Answers2