0

Eurostat data can be downloaded via a REST API. The response format of the API is a XML file formatted according to the SDMX-ML standard. With SAS, very conveniently, one can access XML files with the libname statement and the XML or XMLv2 engine.

Currently, I am using the xmlv2 engine together with the automap= option to generate an xmlmap to access the data. It works. But the resulting SAS data sets are very unstructured, and for another data set to be downloaded the data structure might change. Also the request might depend on the DSD-file that Eurostat provides for each database item within a different XML file.

Here comes the code:

%let path = /your/working/directory/;

filename map "&path.map.txt";
filename resp "&path.resp.txt";

proc http 
    URL="http://ec.europa.eu/eurostat/SDMX/diss-web/rest/data/cdh_e_fos/..PC.FOS1.BE/?startperiod=2005&endPeriod=2011" 
    METHOD="GET"
    OUT=resp;
run;quit;

libname resp XMLv2 automap=REPLACE xmlmap=map;

proc datasets;
copy out=WORK in=resp;
run;quit;

With the code above, you can view all downloaded data in your WORK library. Its a mess.

To download another time series change parameters of the URL according to Eurostat's description.

So here is my question

Is there a way to easily generate a xmlmap from a call to the DSD file so that the data are stored in a well structured way?

As the SDMX-ML standard is widely used in public institutions such as the ECB, Eurostat, OECD... I am wondering if somebody has implemented requests to the databases, already. I know about the tool from Banca Italia which uses a javaObject. However, I was wondering if there might be a solution without the javaObject.

Johannes Bleher
  • 321
  • 3
  • 15
  • This still requires a [MCVE]. – svgrafov Nov 27 '17 at 13:51
  • Updated the question. Thought, I've given enough details... sorry. – Johannes Bleher Nov 27 '17 at 13:59
  • If you were to try work up your own you could start with SAS tool "SAS XML Mapper". It will let you look at the xml in a little clearer way than just viewing automap tables copied to work. The Mapper presents the same automap data but with a UI. Now, If you look at the DSD result http://ec.europa.eu/eurostat/SDMX/diss-web/rest/datastructure/ESTAT/DSD_cdh_e_fos you can see there is a very rich conceptual model that you would have to learn and interpret in order to generate the SAS code that could join the automap tables into human reasonable rectangles ready for analytics. – Richard Nov 29 '17 at 21:42
  • Yes, thanks, @Richard I have tried the XML Mapper, but as you said, it is a very rich conceptual model... and the XML Mapper only rubs my nose in it ;D So I hoped that somebody already worked on it... and found some kind of shortcut... – Johannes Bleher Nov 30 '17 at 09:30

0 Answers0