Only one usage of each socket address (protocol/network address/port) is normally permitted

Question

The last few weeks we have been experiencing this error message while using the Azure Search SDK (1.1.1 - 1.1.2) and performing searches.

We consume the Search SDK from internal APIs (deployed as Azure Web Apps) that scale up-down based on traffic (so there could be more than 1 instance of the APIs doing the searches).

Our API queries 5 different indexes and maintains an in-memory copy of the SearchIndexClient object that corresponds to each index, a very simple implementation would look like:

public class AzureSearchService
{
    private readonly SearchServiceClient _serviceClient;

    private Dictionary<string, SearchIndexClient> _clientDictionary;

    public AzureSearchService()
    {
        _serviceClient = new SearchServiceClient("myservicename", new SearchCredentials("myservicekey"));
        _clientDictionary = new Dictionary<string, SearchIndexClient>();
    }

    public SearchIndexClient GetClient(string indexName)
    {
        try
        {
            if (!_clientDictionary.ContainsKey(indexName))
            {
                _clientDictionary.Add(indexName, _serviceClient.Indexes.GetClient(indexName));
            }
            return _clientDictionary[indexName];
        }
        catch
        {
            return null;
        }
    }

    public async Task<SearchResults> SearchIndex(SearchIndexClient client, string text)
    {
        var parameters = new SearchParameters();
        parameters.Top = 10;
        parameters.IncludeTotalResultCount = true;
        var response = await client.Documents.SearchWithHttpMessagesAsync(text, parameters, null, null);
        return response.Body;
    }
}

And the API would invoke the service by:

public class SearchController : ApiController
{
        private readonly AzureSearchService service;

        public SearchController()
        {
            service = new AzureSearchService();
        }


        public async Task<HttpResponseMessage> Post(string indexName, [FromBody] string text)
        {
            var indexClient = service.GetClient(indexName);
            var results = await service.SearchIndex(indexClient, text);
            return Request.CreateResponse(HttpStatusCode.OK, results, Configuration.Formatters.JsonFormatter);              
        }

}

We are using SearchWithHttpMessagesAsync due to a requirement to receive custom HTTP headers instead of the SearchAsync method.

This way we avoid opening/closing the client under traffic bursts. Before using this memory cache (and wrapping each client on a using clause) we would get port exhaustion alerts on Azure App Services.

Is this a good pattern? Could we be receiving this error because of the multiple instances running in parallel?

In case it is needed, the stack trace shows:

System.Net.Http.HttpRequestException: Only one usage of each socket address (protocol/network address/port) is normally permitted service.ip.address.hidden:443


[SocketException:Only one usage of each socket address (protocol/network address/port)is normally permitted service.ip.address.hidden:443]

at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)

at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure,Socket s4,Socket s6,Socket& socket,IPAddress& address,ConnectSocketState state,IAsyncResult asyncResult,Exception& exception)



[WebException:Unable to connect to the remote server]

at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult,TransportContext& context)

at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)

EDIT: We are also receiving this error A connection attempt failed because the connected party did not properly respond after a period of time:

System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond service.ip.address.hidden:443


[SocketException:A connection attempt failed because the connected party did not properly respond after a period of time,or established connection failed because connected host has failed to respond service.ip.address.hidden:443]

at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)

at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure,Socket s4,Socket s6,Socket& socket,IPAddress& address,ConnectSocketState state,IAsyncResult asyncResult,Exception& exception)



[WebException:Unable to connect to the remote server]

at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult,TransportContext& context)

at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)

Answer 1

As implemented in the code in your question, the cache will not prevent port exhaustion. This is because you're instantiating it as a field of the ApiController, which is created once per request. If you want to avoid port exhaustion, the cache must be shared across all requests. To make it concurrency-safe, you should use something like ConcurrentDictionary instead of Dictionary.

The "connection attempt failed" error is likely unrelated.