HttpWebRequest stops working suddenly, No response received after few requests

Question

I’m working with a WPF .net 4.0 Application. I have a search bar. For each search token I need to do 8 http request to 8 separate URLs to get search results. I send 8 requests to server after 400 milliseconds once the user stops typing in search bar. Searching for 6 to 7 search-tokens results comes very nicely. But after that suddenly HttpWebRequest stops working silently. No exception was thrown, no response was received. I'm working with Windows 7, I disabled the firewall too. I don't know where the subsequent http requests are lost.

Can anyone show me lights to fix this issue?

Below is my code for HttpWebRequest call.

public static void SendReq(string url)
{
    // Create a new HttpWebRequest object.
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

    request.ContentType = "application/x-www-form-urlencoded";
    request.Proxy = new WebProxy("192.168.1.1", 8000);

    // Set the Method property to 'POST' to post data to the URI.
    request.Method = "POST";

    // start the asynchronous operation
    request.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), request);

}

private static void GetRequestStreamCallback(IAsyncResult asynchronousResult)
{
    HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;

    // End the operation
    Stream postStream = request.EndGetRequestStream(asynchronousResult);

    string postData = this.PostData;

    // Convert the string into a byte array. 
    byte[] byteArray = Encoding.UTF8.GetBytes(postData);

    // Write to the request stream.
    postStream.Write(byteArray, 0, byteArray.Length);
    postStream.Close();

    // Start the asynchronous operation to get the response
    request.BeginGetResponse(new AsyncCallback(GetResponseCallback), request);
}

private static void GetResponseCallback(IAsyncResult asynchronousResult)
{
    HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;

    // End the operation
    using(HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asynchronousResult))
    {
        using(Stream streamResponse = response.GetResponseStream())
        {
             using(StreamReader streamRead = new StreamReader(streamResponse))
             {
                 string responseString = streamRead.ReadToEnd();
                 Debug.WriteLine(responseString);
             }
        }
    }
}

How certain are you that no exceptions are being thrown for the other requests? Your `Close` calls aren't in `using` statements, which means if there are exceptions, you'll be leaving response connections open, which could deadlock later requests... — Jon Skeet, Dec 21 '12 at 08:27
No exception is thrown. Application is completely silent. No response received GetResponseCallback is never being called after 6 to 7 requests. I have updated the code. Added *using* but still same issue. — Somnath, Dec 21 '12 at 10:12
Have you looked at what's going on at the network level with something like Wireshark? — Jon Skeet, Dec 21 '12 at 11:22
What does it mean that you read Console input in `GetRequestStreamCallback`? Are you really entering something before sending a request? — Clemens, Dec 21 '12 at 11:26
@Clemens, Yes I'm posting very little amount of data too. I have few post parameters which need to send to server. I have a wrapper class of this code which supplies postData to this code. — Somnath, Dec 21 '12 at 12:03
Doesn't the console input block in `GetRequestStreamCallback`? — Clemens, Dec 21 '12 at 13:00
@JonSkeet, I didn't try Wireshark but I have tried Fiddler, Fiddler is able to catch few request at the beginning. But later on Fiddler is also not receiving any requests being proxy. That means something is blocking my requests. — Somnath, Dec 21 '12 at 13:10
@Somnath: That's what I'd expect if the responses weren't being closed properly. It's odd that it's happening when you *are* closing them. Hmm. Can you easily test a synchronous version? — Jon Skeet, Dec 21 '12 at 14:24
@JonSkeet, right its a good idea to test synchronous version. I will try that soon. — Somnath, Dec 24 '12 at 06:19

Arjun Vachhani · Answer 1 · 2014-03-23T10:31:31.923

I think i am very late, but i still want to answer your question, may be it can be helpful to others. By default HTTP Request you make are HTTP 1.1 requests. And HTTP 1.1 Request by default has Keep-Alive connection. so when you make too many request to same server .net framework only make x no. of request.

you should close all your response by response.Close()

you can also specify how many simultaneous requests you can make.

ServicePointManager.DefaultConnectionLimit = 20;

Note that you have to set DefaultConnectionLimit before the first request you make. you can find more information here on msdn.

score 2 · Answer 2 · answered Dec 21 '12 at 13:09

2

All i can see is that in GetRequestStreamCallback you should replace

postStream.Write(byteArray, 0, postData.Length);

by

postStream.Write(byteArray, 0, byteArray.Length);

since these length aren't necessarily equal.

answered Dec 21 '12 at 13:09

Clemens

123,504
12
155
268

Thanks Clemens, you sounds right. Just now I did the changes in my code. Even it does not solve this problem of silent failure of HttpWebRequest. – Somnath Dec 21 '12 at 13:16

score 0 · Answer 3 · answered Feb 02 '13 at 20:07

0

@Somnath I am not sure if you found this answer, but if anyone else stumbles across this post with the same issue that Somnath and I were having.

We all try to do our due diligence in keeping memory clean and clear, but with streams we always will save unexplained issues if we make sure to flush the stream before we close it.

Replace This :

postStream.Write(byteArray, 0, byteArray.Length);
postStream.Close();

With This :

postStream.Write(byteArray, 0, byteArray.Length);
postStream.Flush();
postStream.Close();

answered Feb 02 '13 at 20:07

Jdunn5

108
2
10

Hi Jdunn, Just now I tested your suggestion but failed. I flushed and closed all streams. So that there is no memory leak in my code. Still no luck. Please note that when I keep on typing in search-box I'm sending too may requests before some requests comes back with response. I'm just ignoring the previous responses. Now I guess I need to do something with request cancellation. Whats in your mind? Any guess..! – Somnath Feb 07 '13 at 07:19

score -1 · Answer 4 · answered Feb 14 '13 at 05:49

I followed all suggestion provide by all of you but couldn't stop the silent failure of HTTP request. But I found a workaround. Even myself could not reach to a final conclusion till now.

But my workaround is working well as off now without any failure.

In SendReq(string url) function i have added the following lines of code

System.Net.ServicePointManager.DefaultConnectionLimit = 100; // Just selected a random number for testing greater than 2
System.Net.ServicePointManager.SetTcpKeepAlive(true, 30, 30); // 30 is based on my server i'm hitting
System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;

request.KeepAlive = true;

HttpWebRequest stops working suddenly, No response received after few requests

4 Answers4