XML Reader Performance

Question

I've traced slugishness in my application to the bit of code that's being timed below. I knew that this would be a slow point but each request is taking an average of 1 second. The bit of xml that I'm after is always in the first tag so I don't think it's the download times that are getting me.

Stopwatch stopwatch = new Stopwatch();
XmlTextReader reader = new XmlTextReader("http://steamcommunity.com/id/test?xml=1");

stopwatch.Reset();
stopwatch.Start();

while (reader.Read()) {
 if (reader.Name.Equals("steamID64")) {
  reader.Read();

  stopwatch.Stop();

  time = stopwatch.ElapsedMilliseconds();
  return Convert.ToInt64(reader.Value);
 }
}

Is there a faster way to read the tag that I want or am I being limited by the server I'm downloading the xml files from?

Thanks.

Try XPathNavigator object - http://msdn.microsoft.com/en-us/library/system.xml.xpath.xpathnavigator.aspx — Hari Pachuveetil, Aug 26 '10 at 21:50
@Yuriy, you mean to use reader.close()? I am doing that, just didn't include it in the snippet above. — Radu, Aug 26 '10 at 21:55
I think Yuriy ment was to wrap the declaration with `using`, like so: `using (XmlTextReader reader = new XmlTextReader("http://steamcommunity.com/id/test?xml=1")){...}`. That way it is automatically disposed (also in the event of an exception) :) — Sune Rievers, Nov 15 '10 at 14:23

score 3 · Answer 1 · answered Aug 26 '10 at 21:50

3

I think you are measuring the time it takes to create a connection. To confirm that, you could move the Reset+Start lines up to above the creation of the reader. I expect there will be little or no difference.

If it is the connect time, it is up to the network and there is noting you can do in your code. Maybe you can get some improvement from tweeking your network settings. But that's for another forum.

answered Aug 26 '10 at 21:50

H H

263,252
30
330
514

I moved the stopwatch to surround the declaration of the reader and it reads 0. – Radu Aug 26 '10 at 21:57
1

I consider that confirmation. It figures, XmlReader is the fastest way to read Xml. It is an I/O problem. – H H Aug 26 '10 at 22:00
2

So, the connection isn't made until you start Read()'ing - then you'll suffer TCP handshake followed by TCP slow-start. If you're going to fetch the stream multiple times during app lifetime, see if you can keep a persistent HTTP connection open to the server. – snemarch Aug 26 '10 at 22:03
@snemarch Thanks for the suggestion, I'll look into that - hopefully it'll make multiple lookups bearable. – Radu Aug 26 '10 at 22:05

score 1 · Answer 2 · answered Aug 26 '10 at 21:52

1

Try setting

reader.XmlResolver = null;

answered Aug 26 '10 at 21:52

Jonathan

1,487
11
12

Doesn't seem to be helping though I probably need to do a lot of iterations to make certain. – Radu Aug 26 '10 at 21:57

score 1 · Accepted Answer · edited Aug 26 '10 at 22:03

1

so I don't think it's the download times that are getting me

To confirm this have you considered downloading the file locally and then seeing what the times are like

edited Aug 26 '10 at 22:03

H H

263,252
30
330
514

answered Aug 26 '10 at 21:55

Conrad Frix

51,984
12
96
155

1

No, trying now. Thanks for the suggestion! – Radu Aug 26 '10 at 21:57
Went down from 1000ms to 1ms. Guess I'm screwed. Can't believe I didn't think to test this - thank you! – Radu Aug 26 '10 at 22:02

score 1 · Answer 4 · answered Aug 26 '10 at 22:12

I've compared your method with a different one. Simply download data and find the id by regular expression.

        System.Diagnostics.Stopwatch stopwatch = new System.Diagnostics.Stopwatch();
        stopwatch.Reset();
        stopwatch.Start();

        System.Net.WebClient wc = new System.Net.WebClient();
        string s = wc.DownloadString("http://steamcommunity.com/id/test?xml=1");
        System.Text.RegularExpressions.Regex re = new Regex("\\<steamID64\\>(\\d+)\\</steamID64\\>");
        System.Text.RegularExpressions.Match m = re.Match(s);
        if (m != null && m.Captures.Count != 0) Response.Write("steamID64: " + m.Captures[0].Value + " <br/>");
        stopwatch.Stop();

        long time = stopwatch.ElapsedMilliseconds;
        Response.Write("Time Elapsed (1):" + time.ToString() +" <br/>");

        stopwatch.Reset();
        stopwatch.Start();

        System.Xml.XmlTextReader reader = new System.Xml.XmlTextReader("http://steamcommunity.com/id/test?xml=1");

        stopwatch.Reset();
        stopwatch.Start();

        while (reader.Read())
        {
            if (reader.Name.Equals("steamID64"))
            {
                reader.Read();
                stopwatch.Stop();

                time = stopwatch.ElapsedMilliseconds;
                s = reader.Value;
                break;
            }
        }

        Response.Write("<br/>steamID64: " + s );
        Response.Write("<br/>Time Elapsed (2):" + time.ToString() + " <br/>");

** Result:

steamID64: 76561197991558078 Time Elapsed (1):1572

steamID64: 76561197991558078 Time Elapsed (2):969

XmlReader is better :).

XML Reader Performance

4 Answers4