I have a computer running Windows Server 2012 that I use to run a lot of my applications. Most of them are processing many asynchronous web requests in parallel (using HttpClient in .NET 4.5).
When I check NETSTAT, I see almost all available (assuming 66,536 is the maximum) ports are in use. The piped output from the basic NETSTAT call is a 4mb text file with 64,583 lines. The majority of these are in TIME_WAIT, lots are not attributed to any particular process but labelled as "System Process".
I understand TIME_WAIT is not necessarily a bad thing and my applications do not seem to be throwing exceptions explicitly stating they are unable to complete a request due to no available ports. However, this concerns me and I'm worried it might be causing time outs.
Is this to be expected? Should I reduce the TcpTimedWaitDelay value in the registry? I believe connections are kept in TIME_WAIT for 4 minutes by default. Also is it possible I'm doing something wrong in my code that leaves connections open like this? I reuse the HttpClient object where possible and always dispose of it when finished. Should I be looking out for connection: close headers in responses?
EDIT: Adding code, as requested. Here's the function that makes the request (part of a custom class wrapper for HttpClient);
Public Async Function WebRequestAsync(Url As String, Optional RequestMethod As RequestMethod = RequestMethod.GET, Optional Content As Object = Nothing, _
Optional ContentType As ContentType = ContentType.Default, Optional Accept As String = DefaultAcceptString, _
Optional AdditionalHeaders As NameValueCollection = Nothing, Optional Referer As String = Nothing, _
Optional NoCache As Boolean = False, Optional CustomCookieHandler As Boolean = False, _
Optional Attempts As Integer = 2, Optional CanBeCancelled As Boolean = True) _
As Tasks.Task(Of HttpResponseMessage)
If Attempts < 1 Then Attempts = 1
Dim Method As HttpMethod = Nothing
Select Case RequestMethod
Case Variables.RequestMethod.DELETE : Method = HttpMethod.Delete
Case Variables.RequestMethod.GET : Method = HttpMethod.Get
Case Variables.RequestMethod.OPTIONS : Method = HttpMethod.Options
Case Variables.RequestMethod.POST : Method = HttpMethod.Post
Case Variables.RequestMethod.PUT : Method = HttpMethod.Put
End Select
'prepare message
Dim Message As New HttpRequestMessage(Method, Url)
Message.Headers.ExpectContinue = False
Message.Headers.TryAddWithoutValidation("Accept", Accept)
If Referer IsNot Nothing Then Message.Headers.Add("Referer", Referer)
If NoCache Then
Message.Headers.Add("Pragma", "no-cache")
Message.Headers.Add("Cache-Control", "no-cache")
End If
If AdditionalHeaders IsNot Nothing Then
For Each Key In AdditionalHeaders.AllKeys
Message.Headers.TryAddWithoutValidation(Key, AdditionalHeaders(Key))
Next
End If
'set content
If Content IsNot Nothing Then
Dim ContentTypeString As String = GetEnumDescription(ContentType)
Dim ContentBytes As Byte() = Nothing
If TypeOf Content Is String Then
ContentBytes = Encoding.UTF8.GetBytes(CType(Content, String))
ElseIf TypeOf Content Is Byte() Then
ContentBytes = CType(Content, Byte())
ElseIf TypeOf Content Is MultiPartPostData Then
Dim MultiPartPostData As MultiPartPostData = CType(Content, MultiPartPostData)
ContentBytes = MultiPartPostData.Bytes
ContentTypeString += "; boundary=" & MultiPartPostData.Boundary
End If
Dim ByteArrayContent As New ByteArrayContent(ContentBytes)
ByteArrayContent.Headers.Add("Content-Type", ContentTypeString)
Message.Content = ByteArrayContent
End If
'get response
Output(RequestMethod.ToString & " " & Url, OutputType.Debug)
'Set cancellation token
Dim CToken As New CancellationToken
If CancellationToken IsNot Nothing AndAlso CancellationToken.HasValue AndAlso CanBeCancelled Then CToken = CancellationToken.Value
Dim Response As HttpResponseMessage = Nothing
For Attempt = 1 To Attempts
Try
Response = Await Client.SendAsync(Message, HttpCompletionOption.ResponseHeadersRead, CToken).ConfigureAwait(False)
Catch ex As Tasks.TaskCanceledException
If DebugMode Then Output(Method.ToString & " " & Url & " Timed out", OutputType.Error)
Catch ex As HttpRequestException
If ex.InnerException IsNot Nothing Then
If DebugMode Then Output(Method.ToString & " " & Url & " " & ex.InnerException.Message, OutputType.Error)
If ex.InnerException.Message = "Timed out" Then Continue For
Else
If DebugMode Then Output(Method.ToString & " " & Url & " " & ex.Message, OutputType.Error)
End If
Catch ex As Exception
If DebugMode Then Output(Method.ToString & " " & Url & " " & ex.Message, OutputType.Error)
End Try
Exit For
Next
If Response IsNot Nothing Then
Output(Method.ToString & " " & Url & " " & Response.StatusCode & " " & Response.StatusCode.ToString, OutputType.Debug)
If CustomCookieHandler AndAlso Cookies IsNot Nothing Then
Dim Values As IEnumerable(Of String) = Nothing
If Response.Headers.TryGetValues("Set-Cookie", Values) Then ManuallyExtractCookies(Values, New Uri(Url).GetLeftPart(UriPartial.Authority))
End If
End If
Return Response
End Function
And an example of making a request;
Using HttpClient As New HttpClientWrapper(User, Task)
Dim Response As String = Await HttpClient.WebRequestStringAsync("http://www.google.com")
If Response Is Nothing Then Return Nothing
End Using
