how to parse sdmx file in c#?

Question

I am trying to parse the following page:

http://sdw-wsrest.ecb.europa.eu/service/data/YC/B.U2.EUR.4F.G_N_A.SV_C_YM.BETA0+BETA1+BETA2+BETA3+TAU1+TAU2?lastNObservations=1

in order to get the value of BETA0, BETA1, etc....

I am struggling as it appears the webpage is in SDMX format and not regular XML. If anyone can help with a c# snippet it would be greatly appreciated.

Thank you in advance.

SDMX-ML is XML according to this link: https://sdmx.org/?page_id=5008 — David Tansey, Jan 25 '17 at 00:42
@JohnKoerner, I have tried to use nugget sdmxsource to parse the sdmx file but without success. I have also tried to use default .Net xml library but here again without success. — akasolace, Jan 25 '17 at 07:10

duraid · Accepted Answer · 2017-01-26T21:50:16.060

You can use the SDMXSource library which is available for Java and .NET.

Otherwise, if you retrieve the same data using the simpler structure specific format rather than the generic format, you can possibly parse the data easier similar to a CSV format. In order to get the structure specific format set the Accept header to 'application/vnd.sdmx.structurespecificdata+xml;version=2.1'.

You might find this cheat sheet helpful.

UPDATE:

Here is an example of parsing the strcuture specific format to produce a key/value record without using a library. It does not cover all the possible cases (for example dataset level attributes) but it's a good start and works for this particular data message.

XML:

<message:DataSet data:action="Replace" data:validFromDate="2017-01-25T22:31:14.760+01:00" data:structureRef="ECB_FMD2" data:dataScope="DataStructure" xsi:type="ecb_fmd2:DataSetType">
    <Series FREQ="B" REF_AREA="U2" CURRENCY="EUR" PROVIDER_FM="4F" INSTRUMENT_FM="G_N_A" PROVIDER_FM_ID="SV_C_YM" DATA_TYPE_FM="BETA0" COLLECTION="E" TITLE_COMPL="Euro area (changing composition) - Government bond, nominal, all issuers whose rating is triple A - Svensson model - continuous compounding - yield error minimisation - Yield curve parameters, Beta 0 - Euro, provided by ECB" DECIMALS="6" UNIT="PURE_NUMB" TITLE="Yield curve parameters, Beta 0 - Government bond, nominal, all issuers whose rating is triple A - Euro area (changing composition)" UNIT_MULT="0">
        <Obs TIME_PERIOD="2017-01-24" OBS_VALUE="1.775611976078084" OBS_STATUS="A" OBS_CONF="F"/>
    </Series>
    <Series FREQ="B" REF_AREA="U2" CURRENCY="EUR" PROVIDER_FM="4F" INSTRUMENT_FM="G_N_A" PROVIDER_FM_ID="SV_C_YM" DATA_TYPE_FM="BETA1" COLLECTION="E" TITLE_COMPL="Euro area (changing composition) - Government bond, nominal, all issuers whose rating is triple A - Svensson model - continuous compounding - yield error minimisation - Yield curve parameters, Beta 1 - Euro, provided by ECB" DECIMALS="6" UNIT="PURE_NUMB" TITLE="Yield curve parameters, Beta 1 - Government bond, nominal, all issuers whose rating is triple A - Euro area (changing composition)" UNIT_MULT="0">
        <Obs TIME_PERIOD="2017-01-24" OBS_VALUE="-2.438611976090857" OBS_STATUS="A" OBS_CONF="F"/>
    </Series>
    <Series FREQ="B" REF_AREA="U2" CURRENCY="EUR" PROVIDER_FM="4F" INSTRUMENT_FM="G_N_A" PROVIDER_FM_ID="SV_C_YM" DATA_TYPE_FM="BETA2" COLLECTION="E" TITLE_COMPL="Euro area (changing composition) - Government bond, nominal, all issuers whose rating is triple A - Svensson model - continuous compounding - yield error minimisation - Yield curve parameters, Beta 2 - Euro, provided by ECB" DECIMALS="6" UNIT="PURE_NUMB" TITLE="Yield curve parameters, Beta 2 - Government bond, nominal, all issuers whose rating is triple A - Euro area (changing composition)" UNIT_MULT="0">
        <Obs TIME_PERIOD="2017-01-24" OBS_VALUE="11.695146022367336" OBS_STATUS="A" OBS_CONF="F"/>
    </Series>       
</message:DataSet>

Code:

class Program
{
    static void Main(string[] args)
    {
        string path = @"data.xml";

        // An XmlReader created from a file on the disk or any stream like web request for example
        using (var reader = XmlReader.Create(path))
        {
            foreach (var item in GetRecords(reader))
            {
                Debug.WriteLine(string.Join(", ", item.Select(i => string.Format("{0}={1}", i.Key, i.Value))));
            }
        }
    }

    private static IEnumerable<Dictionary<string, string>> GetRecords(XmlReader reader)
    {
        Dictionary<string, string> record = null; 
        while (reader.Read())
        {
            if (reader.IsStartElement() && reader.LocalName == "Series")
            {
                record = new Dictionary<string, string>();
                while (reader.MoveToNextAttribute())
                {
                    record.Add(reader.LocalName, reader.Value);
                }
            }
            else if (reader.IsStartElement() && reader.LocalName == "Obs")
            {                   
                while (reader.MoveToNextAttribute())
                {
                    record.Add(reader.LocalName, reader.Value);
                }

                yield return record;
            }
        }
    }
}

Thank you Duraid, I will try to change the Accept header. Regarding SDMXSource: 1. the docs are not showing a practical usage example for retriving data 2. it seems really a too big library for my little purpose As an alternative, I am also considering to use regex. Thank you.. — akasolace, Jan 26 '17 at 07:43
You're welcome. If not using a library I would use an XmlReader to iterate though the Series and Obs nodes and then read the attributes for these xml nodes and checking the attribute name and value. — duraid, Jan 26 '17 at 14:51

how to parse sdmx file in c#?

1 Answers1