Resolving any hostname takes 12 seconds on a new Windows 2019 server on baremetal, but only when using the system-default DNS servers instead of directly querying one.
I can reproduce it reliably with these powershell commands:
#these are fast
Resolve-DnsName google.com -server 10.0.80.11
Resolve-DnsName google.com -server 10.0.80.12
#this takes 12 seconds before it prints out the response
Resolve-DnsName google.com
The hostname being resolved doesn't matter, and in this example 10.0.80.11 and 10.0.80.12 happen to be the two DNS servers set on both network interfaces:
PS C:\Users\Administrator> Get-DnsClientServerAddress
InterfaceAlias Interface Address ServerAddresses
Index Family
-------------- --------- ------- ---------------
PublicNetwork-Teamed 13 IPv4 {10.0.80.11, 10.0.80.12}
PublicNetwork-Teamed 13 IPv6 {}
PrivateNetwork-Teamed 7 IPv4 {10.0.80.11, 10.0.80.12}
PrivateNetwork-Teamed 7 IPv6 {}
Loopback Pseudo-Interface 1 1 IPv4 {}
Loopback Pseudo-Interface 1 1 IPv6 {}
Nslookup doesn't show this behavior (it's always fast), but that uses its own way of resolving DNS that differ from what the OS actually does.
Using ping or internet explorer (or any other program) with hostnames not yet in the local dns cache will show the same 12 seconds delay.
The server has two teamed network connections, one for a private network (that acts as a 10.0.0.0/8 through vlan magic) and one for the public internet. Both connections are set up with static IP and DNS addresses. It doesn't matter which DNS servers are set.
What I've tried to troubleshoot:
- windows update (from 2019 RTM, to 2019 cumulative update August 2020, to 2019 preview of cumulative update August 2020)
- tried setting other DNS servers (from the IBM Cloud DNS, our own private DNS, and public DNS like 8.8.8.8)
- disabled the "IBM USB Remote NDIS Network Device" - no idea what it's for, maybe IPMI?
- updated the network card (Intel X710-T4) drivers
- updated the network card firmware
- tried disabling IPv6
- disabled the firewall
- disabled windows defender
- rebooted multiple times
- prayed
PS C:\Users\Administrator> Get-DnsClient
InterfaceAlias Interface ConnectionSpecificSuffix ConnectionSpecificSuffix RegisterThisConn UseSuffixWhen
Index SearchList ectionsAddress Registering
-------------- --------- ------------------------ ------------------------ ---------------- -------------
PublicNetwork-Teamed 13 {} True False
PrivateNetwork-Teamed 7 {} True False
Loopback Pseudo-Interface 1 1 {} True False
PS C:\Users\Administrator> ipconfig /all
Windows IP Configuration
Host Name . . . . . . . . . . . . : doomed
Primary Dns Suffix . . . . . . . : example.net
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : example.net
Ethernet adapter PublicNetwork-Teamed:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft Network Adapter Multiplexor Driver
Physical Address. . . . . . . . . : 3C-FD-FE-01-23-45
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 1234:5678:90ab:cd::3(Preferred)
IPv6 Address. . . . . . . . . . . : 1234:5678:90ab:cd:ef12:3456:7890:abcd(Preferred)
Link-local IPv6 Address . . . . . : fe80::b154:f38e:26ae:1e71%13(Preferred)
IPv4 Address. . . . . . . . . . . : 1.2.3.4(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.240
Default Gateway . . . . . . . . . : 1234:5678:90ab:cd::1
1.2.3.1
DHCPv6 IAID . . . . . . . . . . . : 305987070
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-26-E0-4F-3A-3C-FD-FE-83-CB-41
DNS Servers . . . . . . . . . . . : 10.0.80.11
10.0.80.12
NetBIOS over Tcpip. . . . . . . . : Enabled
Ethernet adapter PrivateNetwork-Teamed:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft Network Adapter Multiplexor Driver #2
Physical Address. . . . . . . . . : 3C-FD-FE-01-23-44
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::e591:3764:3ca2:7734%7(Preferred)
IPv4 Address. . . . . . . . . . . : 10.1.2.3(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.192
Default Gateway . . . . . . . . . :
DHCPv6 IAID . . . . . . . . . . . : 222100990
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-26-E0-4F-3A-3C-FD-FE-83-CB-41
DNS Servers . . . . . . . . . . . : 10.0.80.11
10.0.80.12
NetBIOS over Tcpip. . . . . . . . : Enabled
Any idea what's going on or what I should look for next? Thanks!
