Does anyone know why I receive an HttpWebRequest Timeout?

Question

I was wondering if you can help me with a bug I ma having. I have a HTTP Manager I have created that helps me dealing with POSTing/GETing data from websites. It has worked fine until recently when I am trying to use a mixture of both. First loop round everything works, on the second loop it hangs on HttpWebRequest.GetRequestStream(). I have read all over the net and have found no real solution. Below are the codeblocks for the fetching/receiving:

 ASCIIEncoding encoding = new ASCIIEncoding();
 byte[] buffer = encoding.GetBytes(_PostData);

_HttpWebRequest = (HttpWebRequest)WebRequest.Create(_FetchUrl);
_HttpWebRequest.Credentials = _Credentials;
_HttpWebRequest.Method = _RequestType.ToString();
_HttpWebRequest.ContentType = "application/x-www-form-urlencoded";
_HttpWebRequest.ContentLength = buffer.Length;
_HttpWebRequest.UserAgent = userAgent;
_HttpWebRequest.CookieContainer = _CookieContainer;
_HttpWebRequest.KeepAlive = false;
_HttpWebRequest.AllowAutoRedirect = _AllowAutoRedirect;
_HttpWebRequest.AutomaticDecompression = DecompressionMethods.GZip;
_HttpWebRequest.ServicePoint.Expect100Continue = false;  

 if (_RequestType.Equals(RequestTypes.POST))
{
     // Write POST
 Stream reqStream = _HttpWebRequest.GetRequestStream();
 {
  reqStream.Write(buffer, 0, buffer.Length);
  reqStream.Flush();
  reqStream.Close();
    }
}

And the reponse:

HttpWebResponse httpWebResponse = (HttpWebResponse)_HttpWebRequest.GetResponse();
{
  Stream responseStream = httpWebResponse.GetResponseStream();
  {
    if (_UseGzip)
    {
      if (httpWebResponse.ContentEncoding.ToLower().Contains("gzip"))
      {
        responseStream = new GZipStream(responseStream, CompressionMode.Decompress);
      }
      else
      {
        responseStream = new DeflateStream(responseStream, CompressionMode.Decompress);
      }
    }

    if (responseStream != null)
    {
      StreamReader streamReader = new StreamReader(responseStream);
      {
        try
        {
          _PageContent = streamReader.ReadToEnd();
        }
        finally
        {
          streamReader.Close();
          responseStream.Close();
          httpWebResponse.Close();
        }
      }
    }
    else
    {
      _PageContent = string.Empty;
    }
  }
}
_HttpWebRequest.Abort();

Can anyone see any flaws to why my code is hanging? All streams are being closed, I have set the allowed connections to over 100, I don't understand why this is breaking.

Have you tried using something like Wireshark to see what data is actually being sent? — Foole, Feb 11 '10 at 07:55
When you say that the code works the first time but halts on the second, which http methods do you use during the respective requests? — Anders Fjeldstad, Feb 11 '10 at 08:29
The code hangs on a post, GET's seem to be working fine. When I run the code through burp proxy it works fine for some reason? Maybe the proxy is repairing some of my headers because I don't understand how it should make a difference. — Paul Oakham, Feb 11 '10 at 22:29
try John's answer, i experienced similar problem as you did. disposing the resource solved my problem. — Benny, Feb 21 '10 at 07:55

score 11 · Accepted Answer · answered Feb 21 '10 at 07:19

11

This may be due to the fact that you are not disposing of your WebResponse or streams or StreamReaders:

var request = WebRequest.Create(...);
using (var response = request.GetResponse())
{
    using (var responseStream = response.GetResponseStream())
    {
        using (var reader = new StreamReader(responseStream))
        {
            // use the reader
        }
    }
}

answered Feb 21 '10 at 07:19

John Saunders

160,644
26
247
397

I am Aborting the request (no dispose method is available), and closing all available streams. I have also tried setting all of the objects to null and then calling the GC at the end of each request to dispose of them and this also had no affect. – Paul Oakham Feb 22 '10 at 21:26
2

@Paul: you also need to dispose the `StreamReader`, since it implements `IDisposable`. Same with the `Stream`, which you're not closing if an exception is thrown. That's what the `using` block does for you. It makes sure that its parameter is Disposed, no matter what. As soon as I hear that you code fails on subsequent attempts, I wonder, "what did he _not_ do on the _previous_ attempt". Very often, the answer is, "something didn't get Dispose'd". – John Saunders Feb 22 '10 at 21:36

score 2 · Answer 2 · answered Jun 22 '11 at 12:30

I had the same problem. I closed (disposed) all streams and the HttpWebResponse correctly with these using blocks. The problem still persisted when i spammed requests which where aborted by ThreadAbortExceptions. Finally it helped to call myWebRequest.Abort() when the ThreadAbortException occurred! Hope this helps.

score 0 · Answer 3 · answered Nov 27 '13 at 07:58

I see that you use:

HttpWebRequest.AutomaticDecompression = DecompressionMethods.GZip;

Along with manual decompression approach:

      if (httpWebResponse.ContentEncoding.ToLower().Contains("gzip"))
      {
        responseStream = new GZipStream(responseStream, CompressionMode.Decompress);
      }
      else
      {
        responseStream = new DeflateStream(responseStream, CompressionMode.Decompress);
      }

Didn't try if it really matters, but i use only manual approach and in similar code, and it works fine. Actually i have hanging problem, but only with one domain, experimenting with properties showed difference.

Oh, and I suspect if you dont use any post data compression, you need to remove content encoding header

Does anyone know why I receive an HttpWebRequest Timeout?

3 Answers3

Linked