Python BS4 with SDMX

Question

I would like to retrieve data given in a SDMX file (like https://www.bundesbank.de/cae/servlet/StatisticDownload?tsId=BBK01.ST0304&its_fileFormat=sdmx&mode=its). I tried to use BeautifulSoup, but it seems, it does not see the tags. In the following the code

import urllib2
from bs4 import BeautifulSoup 
url = "https://www.bundesbank.de/cae/servlet/StatisticDownload?tsId=BBK01.ST0304&its_fileFormat=sdmx"
html_source = urllib2.urlopen(url).read()
soup = BeautifulSoup(html_source, 'lxml')
ts_series = soup.findAll("bbk:Series")

which gives me an empty object.

Is BS4 the wrong tool, or (more likely) what am I doing wrong? Thanks in advance

The supplied url says 'Your request could not be processed!', maybe just paste some xml snippet would help. — flyingfoxlee, Sep 16 '16 at 13:37
You're right, but the url is readable in python, at least on my system — Daniel, Sep 16 '16 at 13:49

flyingfoxlee · Accepted Answer · 2016-09-16T14:42:59.263

0

soup.findAll("bbk:series") would return the result.

In fact, in this case, even you use lxml as the parser, BeautifulSoup still parse it as html, since html tags are case insensetive, BeautifulSoup downcases all the tags, thus soup.findAll("bbk:series") works. See Other parser problems from the official doc.

If you want to parse it as xml, use soup = BeautifulSoup(html_source, 'xml') instead. It also uses lxml since lxml is the only xml parser BeautifulSoup has. Now you can use ts_series = soup.findAll("Series") to get the result as beautifulSoup will strip the namespace part bbk.

edited Sep 16 '16 at 14:42

answered Sep 16 '16 at 13:58

flyingfoxlee

1,764
1
19
29

Oh dayum. Thank you so much. You saved my day after 2 hours useless testing :) – Daniel Sep 16 '16 at 14:05

Python BS4 with SDMX

1 Answers1