c# Code optimization for large volume of data

Question

I am getting large volume data over TCP. There are 2 type of XML packets in data. I need to process it as fast as possible.

<?xml version="1.0" encoding="UTF-8"?><xsi:Event> .... [dynamic length data] .... </xsi:Event>

and

<?xml version="1.0" encoding="UTF-8"?><ChannelHeartBeat xmlns="http://schema.broadsoft.com/xsi"/>

Sometime packets doesn't have xml declaration.

This is old code.

 char c = (char)streamReader.Read();
 sb.Append(c);

 if(sb.ToString().EndsWith("</xsi:Event>",StringComparison.OrdinalIgnoreCase))
 { 
     ....
     sb.Clear();
 }

 if(sb.ToString().EndsWith("<ChannelHeartBeat xmlns=\"http://schema.broadsoft.com/xsi\"/>", StringComparison.OrdinalIgnoreCase))
 {
   ....
   sb.Clear();

 }

ToString() was taking 26% of CPU time in above code.

Below is optimized code. It enhanced performance by 30%

  char c = (char)streamReader.Read();
  sb.Append(c);
  n++;
  if (n > 60)
  {
      if (c == '>')
      {          
          if (n < 105)
          {
              string temp = sb.ToString();
              if (temp.EndsWith("<ChannelHeartBeat xmlns=\"http://schema.broadsoft.com/xsi\"/>", StringComparison.OrdinalIgnoreCase))
              {
               sb.Clear();
               n = 0;
              }
          }
          if (n > 700)
          {
              string temp = sb.ToString();
              if (temp.EndsWith("</xsi:Event>", StringComparison.OrdinalIgnoreCase))
              {
                 sb.Clear();
                 n = 0;
              }
          }  
      }
  }
}

ToString() is now taking 8% of CPU time.

I want to optimize code further. Any suggestion is welcome.

Thanks in advance.

You can parallelize your code, in order to process xml in parallel. — Jepessen, Dec 18 '17 at 10:32
You can try use the `StringBuilder.ToString(int startIndex, int length)` function to convert only a part of the string _(e.g. first 105 characters or last 50 characters)_. Then the operations should be at least little bit faster. — Julo, Dec 18 '17 at 10:38
Do not call `sb.ToString()` multiple times, constructing string out of chunks is expensive. Using an array-based buffer instead of `StringBuilder` or `List` may help too (check performance to confirm). — Sinatr, Dec 18 '17 at 10:40
Declaration => StringBuilder ss = new StringBuilder(); I am checking with Array buffer. — Manjoor, Dec 18 '17 at 10:48
You can check if string builder ends with some string using loop and indexer (`sb[i]`), avoiding `ToString` completely — Evk, Dec 18 '17 at 10:50
I think the worker thread most of the time waits for data over TCP. You should use asynchronous methods. — Alexander Petrov, Dec 18 '17 at 13:18
I'm curious: did you try my answer? And if yes - did it improve perfomance or not so much? — Evk, Dec 20 '17 at 14:09

Evk · Answer 1 · 2017-12-18T11:19:55.403

You can try to check if string builder ends with given string using loop and indexer and see if that improves perfomance. For example:

public static class Extensions {
    public static bool EndsWith(this StringBuilder sb, string target, bool caseInsensetive = false) {
        // if sb length is less than target string
        // it cannot end with it
        if (sb.Length < target.Length)
            return false;
        var offset = sb.Length - target.Length;            
        for (int i = sb.Length - 1; i >= offset; i--) {
            var left = sb[i];
            var right = target[i - offset];
            // conver to upper-case for insensetive comparision
            // if necessary
            if (caseInsensetive) {
                left = Char.ToUpper(left, CultureInfo.InvariantCulture);
                right = Char.ToUpper(right, CultureInfo.InvariantCulture);
            }
            // fail fast
            if (left != right)
                return false;
        }
        return true;
    }
}

Usage:

const string HeartBeatEnding = "<ChannelHeartBeat xmlns=\"http://schema.broadsoft.com/xsi\"/>";
const string EventEnding = "</xsi:Event>";

char c = (char)streamReader.Read();
sb.Append(c);
if (sb.EndsWith(EventEnding)) {
    // do stuff
}
else if (sb.EndsWith(HeartBeatEnding)) {
    // do stuff
}

I tried with different data structure, now using List and It works perfectly. — Manjoor, Dec 20 '17 at 16:33
Good to know (though internally StringBuilder is also list of chars). — Evk, Dec 20 '17 at 16:41

c# Code optimization for large volume of data

1 Answers1