3

I have one request for a specific domain that take a really long time to complete : 22 seconds in average. The request itself does not return a lot of data.

var httpClient = new HttpClient();  //instantiated at app start and reused 
var request = new HttpRequestMessage(HttpMethod.Get, "http://www.somedomain.com");
var result = await httpClient.SendAsync(request); //take a really long time

After some debugging, it seems related to DNS. If I run the exact same request but replace domain name by IP, it's fast (< 200 ms).

It's also fast if I run the request a second time using same HttpClient instance. However, if I run that second request after more than one minute it's slow again (probably because connections are closed automatically after 1 minute).

Is there a way to improve this ? (eg: to make sure resolved DNS entries would stay in cache for a long period). Also why is DNS so slow for that domain when using HttpClient ? If I try to access that domain using Chrome it's blazing fast (even after flushing DNS and clearing cache). A DNS lookup of that domain using an online service is also very fast. Reported TTL is 60 minutes.

EDIT: I have tried to increase ServicePointManager.DnsRefreshTimeout (which is set by default to 2 minutes). But it does not help.

Calling Dns.GetHostEntry("somedomain.com") is instantaneous and return the right IP. I don't know why request with HttpClient are so slow.

tigrou
  • 4,236
  • 5
  • 33
  • 59
  • 1
    check this thread https://github.com/dotnet/runtime/issues/18348, it is long. If you are using .NET Core, check SocketsHttpHandler.PooledConnectionLifetime – Anand Sowmithiran Mar 27 '22 at 17:23
  • See also "Consider using Happy Eyeballs or similar in SocketsHttpHandler" https://github.com/dotnet/runtime/issues/26177 – Jeremy Lakeman Mar 28 '22 at 03:03

3 Answers3

3

I found out what is wrong: the domain that is slow has an IPV6 address in the DNS records. This can be confirmed by calling this function:

Dns.GetHostEntry("somedomain.com")

First it tries to connect using IPV6. Eventually, a timeout occurs. Then it tries IPV4 (and it works). This explains why it is so slow. The solution I have found so far is to force IPV4 by resolving the domain name myself:

var uri = new Uri("somedomain.com");
var builder = new UriBuilder(uri);
var addresses = await Dns.GetHostAddressesAsync(uri.Host);    
builder.Host = addresses
                 .First(x => x.AddressFamily == AddressFamily.InterNetwork) //IPV4 only
                 .ToString(); 
var request = new HttpRequestMessage(HttpMethod.Get, builder.Uri);
request.Host = uri.Host;

Maybe there is a better way. I can't use SocketsHttpHandler since i'm stuck with .NET Framework 4.5

tigrou
  • 4,236
  • 5
  • 33
  • 59
  • I know this comment is a little old, but there is a .net standard backport: https://www.nuget.org/packages/StandardSocketsHttpHandler/ – Michael Sabin Jun 14 '23 at 14:00
0

It's not only about caching DNS query. If you create new client for every request, then before every request both TCP and HTTP handshakes are made -- which makes your single request slow. This is probably the reason why subsequent requests with "old" agent are faster -- handshakes are only made before first request, but not before second one.

Hrca12
  • 23
  • 1
  • 6
  • Did you saw the comment in my code ? "HttpClient instance is instantiated at app start and reused between subsequent requests". It does not help in my case because 30 minutes will elapse between those requests and after 1 minute, it's slow again. – tigrou Mar 27 '22 at 17:15
  • Read code before commenting – Michiluki Mar 27 '22 at 18:26
0

An other working solution using SocketsHttpHandler and HttpClient here : https://stackoverflow.com/a/70475741/1529139

It uses IPv4 over IPv6 at connection time.

It did the job for me

56ka
  • 1,463
  • 1
  • 21
  • 37