C# Optimize HttpWebRequest

Question

I'm making my own HTTPWebRequest class and I'm trying to optimize the time it takes to read the buffer and make a request, so far it takes around 800-1000ms for every request to google.com

Same code in C++ takes ~ 200ms any suggestions how I could optimize this better?

public class OWebReq
{
public string LastError;
public bool LastOK = false;
public OWebReq()
{

}
public static void Init()
{
    ServicePointManager.Expect100Continue = false;
    ServicePointManager.DefaultConnectionLimit = 9999;
    WebRequest.DefaultWebProxy = (WebProxy)null;
    ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
}
public string GET(string url, WebProxy proxy = null)
{
    LastOK = false;
    var req = (HttpWebRequest)WebRequest.Create(url);
    req.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36";
    req.Proxy = proxy;
    req.KeepAlive = false;
    req.Timeout = 5000;
    req.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
    try
    {
        using(var response=(HttpWebResponse)req.GetResponse())
        {
            var responseStream = response.GetResponseStream();
            if(responseStream!=null)
            {
                using (var sr = new StreamReader(responseStream))
                {
                    return sr.ReadToEnd();
                }

            }
        }

    }
    catch (Exception ex)
    {
        LastError = ex.Message;
        return null;
    }
     return null;
 }
}

Executing code till var response=(HttpWebResponse)req.GetResponse() takes ~ 350ms

Have you looked at what's happening in Wireshark or something similar, to see what the situation is at the network level? — Jon Skeet, May 29 '14 at 15:25
For me (I live in the netherlands) I get a "302 Moved" response to google.nl - if you also get a redirect, does the C++ example follow the new url? — C.Evenhuis, May 29 '14 at 17:44

score 0 · Answer 1 · answered Mar 10 '15 at 08:58

Try setting ServicePointManager.Expect100Continue or req.ServicePoint.Expect100Continue to false.

The basic gist is, that the request initially only sends the headers and waits for the server to respond with code 100 (continue). This is capped at 350ms. After the response is received the body of the request is sent. This way you don't put a very huge request on the wire (with a body of many MB), when the server will deny the request anyway (redirect or authentication required, or something wrong in the headers, etc).

This link has a very detailed explanation.

C# Optimize HttpWebRequest

1 Answers1