Why does name resolution not work on the server itself?

Question

I have got an Ubuntu Linux 12.04.1 server which as one service runs bind9 for name resolution for a xxx.local domain which forwards all other requests to the name servers of my Internet provider.

This works fine for all clients in the network (Windows and Linux), they get name resolution as expected.

Until yesterday the same was true for the server itself. But I updated several packages and rebooted it. Everything seemed fine so I went home. This morning, the proxy and email no longer worked. It turned out that name resolution was the culprit. nslookup on the clients still works fine but on the server it always times out:

nslookup google.com
connection timed out; no servers could be reached

nslookup google.com 127.0.0.1
;; connection timed out; no servers could be reached

nslookup google.com 192.168.115.250
;; connection timed out; no servers could be reached

(where 192.168.115.250 is IP address of the server itself)

On a different Linux computer it works:

nslookup google.com
Server:         192.168.115.250
Address:        192.168.115.250#53

Non-authoritative answer:
Name:   google.com
Address: ....

On a Windows XP computer it looks like this:

nslookup google.com
*** Der Servername für die Adresse 192.168.115.250 konnte nicht gefunden werden:
    Non-existent domain
*** Die Standardserver sind nicht verfügbar.
Server:  UnKnown
Address:  192.168.115.250

Nicht autorisierte Antwort:
Name:    google.com
Address:  ...

(The error means "The server name for the address ... could not be found... which probably means that reverse lookup does not work. I never saw that one before.)

If I specify one of the name servers of my Internet provider on my server, it works:

nslookup google.com [IP goes here]
Server:         [IP goes here]
Address:        [IP goes here]#53

Non-authoritative answer:
Name:   google.com
Address: ....

I can't figure out what might be wrong here. Since everything worked fine until yesterday, it must have something to do with the latest Ubuntu update or the reboot.

Any hints would be appreciated.

UPDATE: The firewall should not be the problem:

sudo iptables -L -v -n
Chain INPUT (policy ACCEPT 16M packets, 6138M bytes)
 pkts bytes target     prot opt in     out     source               destination 

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination 

Chain OUTPUT (policy ACCEPT 13M packets, 50G bytes)
 pkts bytes target     prot opt in     out     source               destination

UPDATE2:

Since people keep asking for resolv.conf, I would like to point out, that specifying the IP address of the name server in the nslookup call above did not work. But anyway:

cat /etc/resolv.conf:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 192.168.115.250

(If I change that IP to the name server of my Internet provider, it works, but that's no solution.)

UPDATE3:

netstat -anp | egrep :53
tcp   0   0 192.168.115.250:53    0.0.0.0:*       LISTEN      104414/named
tcp   0   0 127.0.0.1:53          0.0.0.0:*       LISTEN      104414/named
tcp6  0   0 :::53                 :::*            LISTEN      104414/named
udp   0   0 192.168.115.250:53168 5.9.122.148:123 ESTABLISHED 106273/ntpd
udp   0   0 192.168.115.25:53     0.0.0.0:*                   104414/named
udp   0   0 127.0.0.1:53          0.0.0.0:*                   104414/named
udp6  0   0 :::53                 :::*                        104414/named

looks fine to me.

UPDATE4:

ip route show table local
local 127.0.0.0/8 dev lo  proto kernel  scope host  src 127.0.0.1
local 127.0.0.1 dev lo  proto kernel  scope host  src 127.0.0.1
broadcast 192.168.115.0 dev eth0  proto kernel  scope link  src 192.168.115.250
local 192.168.115.250 dev eth0  proto kernel  scope host  src 192.168.115.250
broadcast 192.168.115.255 dev eth0  proto kernel  scope link  src 192.168.115.250

I am not sure how to interpret this.

UPDATE5:

I just found another oddity: It's not just named that does not work but also smtpd:

fetchmail: reading message [...]
Trying to connect to 127.0.0.1/25...connection failed.
fetchmail: connection to localhost:smtp [127.0.0.1/25] failed: Connection timed out.
fetchmail: Connection errors for this poll:
name 0: connection to localhost:smtp [127.0.0.1/25] failed: Connection timed out.
fetchmail: SMTP connect to localhost failed

Have you checked the firewall rules? Do they allow connections to/from `lo:`? — the-wabbit, Aug 23 '13 at 11:20
How do I check that? According to Webmin the firewall allows all traffic. — dummzeuch, Aug 23 '13 at 11:34
I have no idea about what Webmin might or might not be checking. If running `iptables -L -v -n` is showing you the INPUT and OUTPUT chains with ACCEPT as the default policy and no reject rules preventing traffic to/from `lo`, you're fine. If not, just add the appropriate rules: `iptables -I INPUT -i lo -j ACCEPT` and `iptables -I OUTPUT -o lo -j ACCEPT` — the-wabbit, Aug 23 '13 at 11:39
Assuming you are using bind, do you have any views or matches defined? (`egrep '(view|match)' /etc/bind/named.conf*`) — the-wabbit, Aug 23 '13 at 12:00
No, there are not view or match entries in /etc/bind/named.conf* (I have run egrep and also checked those files myself.) — dummzeuch, Aug 23 '13 at 12:16
last but not least: `ip route show table local` contains a `local` route to 192.168.115.250 with the right interface, `ip rule show` just spits out the `local`, `main` and `default` tables, `ip rule show` and `netstat -anp | egrep :53` shows bind listening on 0.0.0.0:53 or 192.168.115.250:53? — the-wabbit, Aug 23 '13 at 12:34

score 3 · Answer 1 · answered Aug 23 '13 at 13:18

3

Found it: For whatever reason the loopback interface didn't come up:

ifconfig
eth0      Link encap:Ethernet  HWaddr 00:1e:67:6e:cd:3b
          inet addr:192.168.115.250  Bcast:192.168.115.255  Mask:255.255.255.0
          inet6 addr: fe80::21e:67ff:fe6e:cd3b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:58326791 errors:0 dropped:4 overruns:0 frame:0
          TX packets:109062010 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21089630035 (21.0 GB)  TX bytes:131207523503 (131.2 GB)
          Interrupt:19 Memory:c1c00000-c1c20000

Notice that there is no "lo" here.

Starting it with ...

ifup -v lo

... fixed the problem.

Thanks to everybody who tried to help, in particular to synneticon-dj.

answered Aug 23 '13 at 13:18

dummzeuch

607
1
9
20

Makes sure your fix survives a reboot by editing `/etc/network/interfaces` or whatever your distribution uses. – mveroone Aug 23 '13 at 14:14
This is *really* unusual, never have seen it before. Glad you found the issue. – the-wabbit Aug 23 '13 at 15:38
1

@Kwaio: Just checked. It's in there. Thanks for the reminder. But that makes it even more curious why this happened. – dummzeuch Aug 26 '13 at 08:05

Why does name resolution not work on the server itself?

1 Answers1