-1

I'm having some issues doing DNS lookups against most DNS servers from the guests on my ESXi 5.5 host. I'm saying most because I can do DNS lookups just fine against the DNS servers I've configured under "DNS and Routing" in the vSphere Client, but against the DNS servers that are not configured the DNS request times out.

Here's an example of a nslookup from the ESXi guest (the IP of the configured DNS server is replaced with x.x.x.x):

user@guest:~$ nslookup example.com
Server:         x.x.x.x
Address:        x.x.x.x#53

Non-authoritative answer:
Name:   example.com
Address: 93.184.216.34

Doing the same nslookup from the same ESXi guest against e.g. Google Public DNS instead results in this:

user@guest:~$ nslookup example.com 8.8.8.8
;; connection timed out; no servers could be reached

If I however SSH into the ESXi host instead and try to do a nslookup from there against Google Public DNS it works just fine, as seem below:

~ # nslookup nslookup example.com 8.8.8.8
Server:    8.8.8.8
Address 1: 8.8.8.8 google-public-dns-a.google.com

Non-authoritative answer:
Name:   example.com
Address: 93.184.216.34

If I run tcpdump on the guest I can see the requests go out, but I only get a response back when I query against x.x.x.x. I've tried to run telnet 8.8.8.8 53 and it connects successfully, so something is explicitly blocking DNS requests. This same issues occurs on all four guests on the machine, so I suspect that ESXi is blocking all requests that aren't against the configured DNS somehow, but I have no idea where or why.

Any ideas what might be preventing the guests from using other DNS servers? Configuring other DNS servers in /etc/resolv.conf doesn't change anything, and neither does changing servers under "DNS and Routing" in vSphere (although I didn't try restarting the host afterwards).

Edit: iptables -S on one of the guest machines:

-P INPUT DROP
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N sshguard
-N ufw-after-forward
-N ufw-after-input
-N ufw-after-logging-forward
-N ufw-after-logging-input
-N ufw-after-logging-output
-N ufw-after-output
-N ufw-before-forward
-N ufw-before-input
-N ufw-before-logging-forward
-N ufw-before-logging-input
-N ufw-before-logging-output
-N ufw-before-output
-N ufw-logging-allow
-N ufw-logging-deny
-N ufw-not-local
-N ufw-reject-forward
-N ufw-reject-input
-N ufw-reject-output
-N ufw-skip-to-policy-forward
-N ufw-skip-to-policy-input
-N ufw-skip-to-policy-output
-N ufw-track-forward
-N ufw-track-input
-N ufw-track-output
-N ufw-user-forward
-N ufw-user-input
-N ufw-user-limit
-N ufw-user-limit-accept
-N ufw-user-logging-forward
-N ufw-user-logging-input
-N ufw-user-logging-output
-N ufw-user-output
-A INPUT -j sshguard
-A INPUT -j ufw-before-logging-input
-A INPUT -j ufw-before-input
-A INPUT -j ufw-after-input
-A INPUT -j ufw-after-logging-input
-A INPUT -j ufw-reject-input
-A INPUT -j ufw-track-input
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -j ufw-before-logging-forward
-A FORWARD -j ufw-before-forward
-A FORWARD -j ufw-after-forward
-A FORWARD -j ufw-after-logging-forward
-A FORWARD -j ufw-reject-forward
-A FORWARD -j ufw-track-forward
-A OUTPUT -j ufw-before-logging-output
-A OUTPUT -j ufw-before-output
-A OUTPUT -j ufw-after-output
-A OUTPUT -j ufw-after-logging-output
-A OUTPUT -j ufw-reject-output
-A OUTPUT -j ufw-track-output
-A ufw-after-input -p udp -m udp --dport 137 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 138 -j ufw-skip-to-policy-input
-A ufw-after-input -p tcp -m tcp --dport 139 -j ufw-skip-to-policy-input
-A ufw-after-input -p tcp -m tcp --dport 445 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 67 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 68 -j ufw-skip-to-policy-input
-A ufw-after-input -m addrtype --dst-type BROADCAST -j ufw-skip-to-policy-input
-A ufw-after-logging-forward -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-after-logging-input -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-before-forward -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 4 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 12 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A ufw-before-forward -j ufw-user-forward
-A ufw-before-input -i lo -j ACCEPT
-A ufw-before-input -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-input -m conntrack --ctstate INVALID -j ufw-logging-deny
-A ufw-before-input -m conntrack --ctstate INVALID -j DROP
-A ufw-before-input -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 4 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 12 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A ufw-before-input -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A ufw-before-input -j ufw-not-local
-A ufw-before-input -d 224.0.0.251/32 -p udp -m udp --dport 5353 -j ACCEPT
-A ufw-before-input -d 239.255.255.250/32 -p udp -m udp --dport 1900 -j ACCEPT
-A ufw-before-input -j ufw-user-input
-A ufw-before-output -o lo -j ACCEPT
-A ufw-before-output -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-output -j ufw-user-output
-A ufw-logging-allow -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW ALLOW] "
-A ufw-logging-deny -m conntrack --ctstate INVALID -m limit --limit 3/min --limit-burst 10 -j RETURN
-A ufw-logging-deny -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-not-local -m addrtype --dst-type LOCAL -j RETURN
-A ufw-not-local -m addrtype --dst-type MULTICAST -j RETURN
-A ufw-not-local -m addrtype --dst-type BROADCAST -j RETURN
-A ufw-not-local -m limit --limit 3/min --limit-burst 10 -j ufw-logging-deny
-A ufw-not-local -j DROP
-A ufw-skip-to-policy-forward -j DROP
-A ufw-skip-to-policy-input -j DROP
-A ufw-skip-to-policy-output -j ACCEPT
-A ufw-track-output -p tcp -m conntrack --ctstate NEW -j ACCEPT
-A ufw-track-output -p udp -m conntrack --ctstate NEW -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 2222 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 80 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 443 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 143 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 993 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 110 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 995 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 587 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 25 -j ACCEPT
-A ufw-user-input -p udp -m udp --dport 123 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 53 -j ACCEPT
-A ufw-user-input -p udp -m udp --dport 53 -j ACCEPT
-A ufw-user-limit -m limit --limit 3/min -j LOG --log-prefix "[UFW LIMIT BLOCK] "
-A ufw-user-limit -j REJECT --reject-with icmp-port-unreachable
-A ufw-user-limit-accept -j ACCEPT
jyggen
  • 1
  • 2
  • Can you provide details on your firewall? – Daniel Widrick Apr 10 '16 at 18:43
  • The ESXi host and guest might be on drastically different networks - testing from the host doesn't really tell us anything (nor does testing TCP). Definitely need firewall and network layout details (most likely someone firewalled the VM guest network to not allow outbound DNS to the internet, forcing lookups to use the network's recursive DNS servers) - that firewall's unlikely to live on the ESXi host directly unless you're using a product to integrate a firewall into your VMware environment (like vShield). – Shane Madden Apr 10 '16 at 18:46
  • I've updated the post with iptables info. What else would be good to supply? – jyggen Apr 10 '16 at 18:53
  • The ESXi host isn't a DNS resolver for the guests. The DNS settings in the ESXi configuration are for the hosts own DNS resolution and has nothing to do with DNS resolution for the guests. You seem to be conflating the two. Additionally, the ESXi host is not a firewall for the guests. The ESXi Security Profile settings are relevant to the host, not the guests. The ESXi host is also not a router for the guests. The routing configuration for the host is applicable to the host, not the guests. – joeqwerty Apr 10 '16 at 21:54

1 Answers1

0

I think you are right, most likely it's the ESX firewall on the host itself, probably it's explicitly blocking the protocol since you are able to successfully connect to the port with telnet. i would start by listing the firewall rules

esxcfg-firewall -s

if it's only listing the DNS server configured in the client, then you should be able to allow all DNS requests manually - It seems presumptuous that it would be be blocking by default - but that's VMware! 

esxcfg-firewall -o 53,tcp,out,tcpdns
Sum1sAdmin
  • 1,934
  • 1
  • 12
  • 20
  • I get `esxcfg-firewall: not found`. Running `esxcli network firewall ruleset rule list` says that dns is allowed outbound on TCP and inbound/outbound on UDP. – jyggen Apr 10 '16 at 19:04
  • if you change the DNS server in 'DNS and routing' does this it have any effect? – Sum1sAdmin Apr 10 '16 at 19:34
  • I don't think so, although I didn't restart the host. You think that's required? – jyggen Apr 10 '16 at 19:54
  • I don't think you should need to reboot, it's weird that your hosts can resolve on the internet and your guests can't - is the virtual network mapped to a real vlan? or are you using some other mode - bridged etc. – Sum1sAdmin Apr 10 '16 at 20:03
  • [This is how our network looks](https://i.imgur.com/NM959Is.png). The DNS server we use is one provided by our data centre and does not "live" of one of the guests, which is why it's weird that it can access any other DNS servers (because they apparently can make it out to the external DNS provided by the data centre). – jyggen Apr 10 '16 at 20:33
  • the data centre DNS will be configured to forward requests up to root servers, I would presume that the hosts are on the management network and there are different switch ACL's and firewall rules at the perimeter. – Sum1sAdmin Apr 10 '16 at 20:44