Configure router/gateway on Ubuntu issue: unable to reach hosts behind ISPs gateway (network map provided)

Question

I try to configure my notebook running Ubuntu 11.04 (desktop edition) as a gateway (may be router needed? --- this is the bit of confusion for me), so that I can share Internet connection (incoming on eth0) via my WiFi card (wlan1). The issue is that at my gateway everything works fine, but from inside local network (behind my notebook) any hosts that are outside of the local ISP's network are unreachable.

Here is the map of my ISP's network: enter image description here

My location is at one of the small PC stations at the bottom right corner of the image. From my gateway I can ping any host, but from PCs connected to it via WiFi I can ping any internal host (i.e. any nodes on the picture below or left of upper switch). As far as I understand the top-level gateway of my ISP is 192.168.1.1 (strictly below of upper switch), which is routed on two different outer ISPs.

That is the barrier I can't fight. Assuming that networks 195.19.50.64/29 and 89.188.114.64/29 have gateways 195.19.50.65 and 89.188.114.65 correspondingly, I can ping them from my gateway, but not from any node connected to it.

Currently my gateway

dvolosnykh@xakac:~$ uname -a
Linux xakac 2.6.38-9-generic #43-Ubuntu SMP Thu Apr 28 15:23:06 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

is configured as follows:

dvolosnykh@xakac:~$ sudo cat /etc/network/interfaces

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 192.168.55.151
    netmask 255.255.255.0
    gateway 192.168.55.1

auto wlan1
iface wlan1 inet static
    address 10.42.43.1
    netmask 255.255.255.0

dvolosnykh@xakac:~$ ifconfig

eth0      Link encap:Ethernet  HWaddr 00:90:f5:8c:4a:ac  
          inet addr:192.168.55.151  Bcast:192.168.55.255  Mask:255.255.255.0
          inet6 addr: fe80::290:f5ff:fe8c:4aac/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:39179 errors:0 dropped:1062 overruns:0 frame:0
          TX packets:26225 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:14947312 (14.9 MB)  TX bytes:4679160 (4.6 MB)
          Interrupt:46 Base address:0x6000 

lo        Link encap:Локальная петля (Loopback)  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1483 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1483 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:264884 (264.8 KB)  TX bytes:264884 (264.8 KB)

mon.wlan1 Link encap:UNSPEC  HWaddr 00-1D-D9-29-00-6C-00-00-00-00-00-00-00-00-00-00  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:11017 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:709779 (709.7 KB)  TX bytes:0 (0.0 B)

wlan1     Link encap:Ethernet  HWaddr 00:1d:d9:29:00:6c  
          inet addr:10.42.43.1  Bcast:10.42.43.255  Mask:255.255.255.0
          inet6 addr: fe80::21d:d9ff:fe29:6c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1746 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1846 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:273576 (273.5 KB)  TX bytes:1283725 (1.2 MB)

dvolosnykh@xakac:~$ iwconfig

lo        no wireless extensions.

eth0      no wireless extensions.

wlan1     IEEE 802.11bg  Mode:Master  Frequency:2.412 GHz  Tx-Power=20 dBm   
          Retry  long limit:7   RTS thr:off   Fragment thr:off
          Power Management:off

mon.wlan1  IEEE 802.11bg  Mode:Monitor  Tx-Power=20 dBm   
          Retry  long limit:7   RTS thr:off   Fragment thr:off
          Power Management:off

Linux kernel is configured to forward packets between interfaces:

dvolosnykh@xakac:~$ sed 's/#.*//;s/^[ \t]*//;s/[ \t]*$//' /etc/sysctl.conf | grep -v '^$'

net.ipv4.ip_forward=1
net.ipv4.ip_dynaddr=1 # do I really need this one?

My iptables configuration:

============================================
                 raw
--------------------------------------------
Chain PREROUTING (policy ACCEPT 2 packets, 120 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1 packets, 52 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
============================================
                 mangle
--------------------------------------------
Chain PREROUTING (policy ACCEPT 2 packets, 120 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 2 packets, 120 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1 packets, 52 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 1 packets, 52 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
============================================
                 nat
--------------------------------------------
Chain PREROUTING (policy ACCEPT 1 packets, 68 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 1 packets, 68 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 MASQUERADE  all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           
============================================
                 filter
--------------------------------------------
Chain INPUT (policy ACCEPT 2 packets, 120 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1 packets, 52 bytes)
num   pkts bytes target     prot opt in     out     source               destination

dvolosnykh@xakac:~$ route -n

Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.55.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.42.43.0      0.0.0.0         255.255.255.0   U     0      0        0 wlan1
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eth0
0.0.0.0         192.168.55.1    0.0.0.0         UG    100    0        0 eth0

Also, my gateway runs hostapd (to bring WiFi card up as Access Point):

dvolosnykh@xakac:~$ cat /etc/hostapd/hostapd.conf

interface=wlan1
driver=nl80211
ssid=xakac
hw_mode=g
channel=1
macaddr_acl=0
auth_algs=1
wpa=3
wpa_passphrase=1234567890
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP

dnsmasq (as DNS and DHCP server):

dvolosnykh@xakac:~$ sed 's/#.*//;s/^[ \t]*//;s/[ \t]*$//' /etc/dnsmasq.conf | grep -v '^$'

domain-needed
bogus-priv
interface=wlan1
dhcp-range=10.42.43.10,10.42.43.254,12h
log-dhcp
log-facility=/var/log/dnsmasq.log

squid (as transparent proxy):

dvolosnykh@xakac:~$ sed 's/#.*//;s/^[ \t]*//;s/[ \t]*$//' /etc/squid3/squid.conf | grep -v '^$'

acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl wifinet src 10.42.43.0/24
acl virbr0net src 192.168.122.0/24
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow wifinet
http_access allow virbr0net
http_access allow localhost
http_access deny all
http_port 3128 transparent
hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:       1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern .       0   20% 4320

Note: I'm able to browse the Web if clients are hardcoded by setting proxy to 10.42.43.1:3128. But if I set client to assume a direct Internet connection (in this case squid is skipped by forwarding), then I am able to see only my ISP's web-pages.

Observation by means of tcpdump gives such results:

1) ping 192.168.1.1 (same with any other host internal to my ISP):

12:40:56.902020 IP 10.42.43.199 > inet.icn.bmstu.ru: ICMP echo request, id 1767, seq 1, length 64
12:40:56.902669 IP inet.icn.bmstu.ru > 10.42.43.199: ICMP echo reply, id 1767, seq 1, length 64
12:40:57.903821 IP 10.42.43.199 > inet.icn.bmstu.ru: ICMP echo request, id 1767, seq 2, length 64
12:40:57.904360 IP inet.icn.bmstu.ru > 10.42.43.199: ICMP echo reply, id 1767, seq 2, length 64

2) ping 89.188.114.65 (same with google.ru, etc.):

12:39:59.516094 IP 10.42.43.199 > 89.188.114.65: ICMP echo request, id 1766, seq 1, length 64
12:40:00.524263 IP 10.42.43.199 > 89.188.114.65: ICMP echo request, id 1766, seq 2, length 64

EDIT:

dvolosnykh@xakac-acer:~$ traceroute6 89.188.114.65

traceroute: unknown host 89.188.114.65

dvolosnykh@xakac-acer:~$ nslookup 89.188.114.65

Server:     10.42.43.1
Address:    10.42.43.1#53

Non-authoritative answer:
65.114.188.89.in-addr.arpa  canonical name = he.hoster.ru.

Authoritative answers can be found from:
hoster.ru
    origin = ns10.hoster.ru
    mail addr = info.filanco.ru
    serial = 2011051902
    refresh = 16384
    retry = 2048
    expire = 1048576
    minimum = 2560

EDIT 2:

I did tracepath (the result is equivalent to mtrs one), they both stuck on 192.168.55.1 (which is the nearest gateway I am provided by ISP to staticly set in eth0s configuration):

dvolosnykh@xakac-acer:~$ tracepath 89.188.114.65
 1:  xakac-acer                                            0.224ms pmtu 1500
 1:  10.42.43.1                                            1.028ms 
 1:  10.42.43.1                                            0.599ms 
 2:  192.168.55.1                                          1.296ms 
 3:  no reply

tcpdump output meanwhile tracepathing:

14:11:45.614219 IP 10.42.43.199.33956 > 89.188.114.65.44444: UDP, length 1472
14:11:45.614313 IP xakac.local > 10.42.43.199: ICMP time exceeded in-transit, length 556
14:11:45.615232 IP 10.42.43.199.54892 > xakac.local.domain: 53509+ PTR? 1.43.42.10.in-addr.arpa. (41)
14:11:45.615386 IP xakac.local.domain > 10.42.43.199.54892: 53509 NXDomain* 0/0/0 (41)
14:11:45.616155 IP 10.42.43.199.33956 > 89.188.114.65.44445: UDP, length 1472
14:11:45.616186 IP xakac.local > 10.42.43.199: ICMP time exceeded in-transit, length 556
14:11:45.616676 IP 10.42.43.199.49164 > xakac.local.domain: 48910+ PTR? 1.43.42.10.in-addr.arpa. (41)
14:11:45.616760 IP xakac.local.domain > 10.42.43.199.49164: 48910 NXDomain* 0/0/0 (41)
14:11:45.621798 IP 10.42.43.199.33956 > 89.188.114.65.44446: UDP, length 1472
14:11:45.622538 IP serv5-2.icn.bmstu.ru > 10.42.43.199: ICMP time exceeded in-transit, length 36
14:11:45.622999 IP 10.42.43.199.47676 > xakac.local.domain: 9789+ PTR? 1.55.168.192.in-addr.arpa. (43)
14:11:45.623084 IP xakac.local.domain > 10.42.43.199.47676: 9789 NXDomain* 0/0/0 (43)
14:11:45.623678 IP 10.42.43.199.33956 > 89.188.114.65.44447: UDP, length 1472
14:11:45.718934 IP6 fe80::21d:d9ff:fe29:6c.mdns > ff02::fb.mdns: 0 PTR (QM)? 65.114.188.89.in-addr.arpa. (44)
14:11:45.719012 IP xakac.local.mdns > 224.0.0.251.mdns: 0 PTR (QM)? 65.114.188.89.in-addr.arpa. (44)
14:11:46.625502 IP 10.42.43.199.33956 > 89.188.114.65.44448: UDP, length 1472
14:11:46.719943 IP6 fe80::21d:d9ff:fe29:6c.mdns > ff02::fb.mdns: 0 PTR (QM)? 65.114.188.89.in-addr.arpa. (44)
14:11:46.720079 IP xakac.local.mdns > 224.0.0.251.mdns: 0 PTR (QM)? 65.114.188.89.in-addr.arpa. (44)

EDIT 3:

Well, we've almoust catched it! Here's output of tcpdump on 192.168.55.151 while pinging 89.188.144.65 from connected client:

On WAN (eth0) interface:

dvolosnykh@xakac:~$ sudo tcpdump -ni eth0 'icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply'
[sudo] password for dvolosnykh: 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
00:24:38.556189 IP 192.168.55.151 > 89.188.114.65: ICMP echo request, id 2174, seq 1, length 64
00:24:39.561864 IP 192.168.55.151 > 89.188.114.65: ICMP echo request, id 2174, seq 2, length 64
00:24:39.627318 IP 89.188.114.65 > 192.168.55.151: ICMP echo reply, id 2174, seq 2, length 64
00:24:40.571398 IP 192.168.55.151 > 89.188.114.65: ICMP echo request, id 2174, seq 3, length 64
00:24:41.570572 IP 192.168.55.151 > 89.188.114.65: ICMP echo request, id 2174, seq 4, length 64
00:24:41.591709 IP 89.188.114.65 > 192.168.55.151: ICMP echo reply, id 2174, seq 4, length 64
00:24:42.577135 IP 192.168.55.151 > 89.188.114.65: ICMP echo request, id 2174, seq 5, length 64
00:24:42.655270 IP 89.188.114.65 > 192.168.55.151: ICMP echo reply, id 2174, seq 5, length 64

On LAN (wlan1) interface:

dvolosnykh@xakac:~$ sudo tcpdump -ni wlan1 'icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply'
[sudo] password for dvolosnykh: 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlan1, link-type EN10MB (Ethernet), capture size 65535 bytes
00:24:38.556105 IP 10.42.43.199 > 89.188.114.65: ICMP echo request, id 2174, seq 1, length 64
00:24:39.561820 IP 10.42.43.199 > 89.188.114.65: ICMP echo request, id 2174, seq 2, length 64
00:24:40.571342 IP 10.42.43.199 > 89.188.114.65: ICMP echo request, id 2174, seq 3, length 64
00:24:41.570546 IP 10.42.43.199 > 89.188.114.65: ICMP echo request, id 2174, seq 4, length 64
00:24:42.577085 IP 10.42.43.199 > 89.188.114.65: ICMP echo request, id 2174, seq 5, length 64

So, incoming from outside replies are not NAT'ed (masqueraded) backwards.

Could you please also provide the output of "traceroute 89.188.114.65". — wolfgangsz, May 26 '11 at 09:22
@wolfgangsz: I've edited my initial post in order to provide you with `traceroute` and `nslookup` outputs. — Dmitrii Volosnykh, May 26 '11 at 09:36
The output of traceroute doesn't make any sense. This looks like it thinks this is a name and needs to be resolved. I would have expected something very different, i.e. a list of hops with latency figures. Can you try to run mtr on that address? — wolfgangsz, May 26 '11 at 10:09
the traceroute output does make sense as it is not traceroute but traceroute6 (note the 6 at the end) that is called, it expects an IPv6 address as argument and not an IPv4 address. Newer versions of Ubuntu started shipping tracepath instead of traceroute (which you seem to have found already). — Marcel G, May 26 '11 at 11:48

score 2 · Accepted Answer · answered May 26 '11 at 10:40

2

Your tracepath output provides the answer. There is something on 192.168.55.1 which prevents the packets from being routed properly or from being transferred. Could be a routing issue, could be firewall. If you have access to that computer, then have a look at these two first.

Seeing that your wireless network hands out addresses in the 10.x.y.z range, and the rest of this network operates on 192.168.a.b, these two are the most likely areas of problems.

answered May 26 '11 at 10:40

wolfgangsz

8,847
3
30
34

@wolfgangsz: I don't have access to 192.168.55.1. How it can be possible that it routes correctly, if I access it directly from my gateway (192.168.55.151), and has some routing issues, when I access it from behind my gateway? It should see, implying masquerading, all packets as sourcing from the only my gateway without any knowledge on the network it may hide... – Dmitrii Volosnykh May 26 '11 at 11:09
1

Since you have access to your gateway (192.168.55.151), can you observe the masquerading with tcpdump? – wolfgangsz May 26 '11 at 11:28
@wolfgangsz: With you help, we're now close to the solution. See **EDIT 3** section. – Dmitrii Volosnykh May 26 '11 at 16:48
@wolfgagsz: Maybe, there's something missing in routing table on my gateway? – Dmitrii Volosnykh May 26 '11 at 18:23
OK, could you replace these tcpdumps with ones that do not show hostnames, so that I can see the actual IP addresses involved. The routing table from above seems fine, but maybe the NAT isn't working as it should. – wolfgangsz May 26 '11 at 20:06
@wolfgangsz: I rewrote **EDIT 3** section. You are welcome. – Dmitrii Volosnykh May 26 '11 at 20:28
I can't see anything wrong in the iptables rules, but clearly the reverse path of the MASQUERADE isn't working. Could you try to replace this rule with another that uses SNAT instead, e.g. iptables -t nat -R POSTROUTING 1 -o eth0 -j SNAT --to-source 192.168.55.151 (to replace the existing rule)? – wolfgangsz May 26 '11 at 21:37
@wolfgangsz: I tried using SNAT insteead of MASQUERADE and nothing have changed. – Dmitrii Volosnykh May 27 '11 at 04:51
@wolfgangsz: I've came across such conversation: https://bbs.archlinux.org/viewtopic.php?id=115681. The problem seems to be similar. But up to now, I didn't fully realize, how virtual interfaces could help me. Continuing researching. – Dmitrii Volosnykh May 27 '11 at 06:48

Dmitrii Volosnykh · Answer 2 · 2011-05-27T14:01:32.077

Running

sudo tcpdump -nvvi eth0 'icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply'

I've discovered that all replies have TTL=1:

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
17:07:58.480713 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.55.151 > 89.188.114.65: ICMP echo request, id 27259, seq 1, length 64
17:07:58.487796 IP (tos 0x0, ttl 1, id 44648, offset 0, flags [DF], proto ICMP (1), length 84)
    89.188.114.65 > 192.168.55.151: ICMP echo reply, id 27259, seq 1, length 64
17:07:59.482058 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.55.151 > 89.188.114.65: ICMP echo request, id 27259, seq 2, length 64
17:07:59.486828 IP (tos 0x0, ttl 1, id 44734, offset 0, flags [DF], proto ICMP (1), length 84)
    89.188.114.65 > 192.168.55.151: ICMP echo reply, id 27259, seq 2, length 64

So, my ISP is so kind to provide me with packets that will live no longer than I need them to process on my PC (which they beleive to be on the very endpoint).

Well, no way :)

iptables --table mangle --append PREROUTING --in-interface eth0 --jump TTL --ttl-inc 1

and I'm going out! Yeah!

Many thanks to @wolfgangsz for his diagnostic copabilities! – Dmitrii Volosnykh May 27 '11 at 13:29 — Dmitrii Volosnykh, May 27 '11 at 13:29

Configure router/gateway on Ubuntu issue: unable to reach hosts behind ISPs gateway (network map provided)

2 Answers2