3

Setup

ubuntu linux VM ( virtual-box) is configured with two interfaces, eth0 and eth1.

eth0 is on a bridged network and directly connected to external network.

eth1 is on "nat network", which is also connected to external network.

Problem

Unable to ping via eth0. Able to connect to the host via TCP.

ping -I eth0  -c2 google.com
PING google.com (172.217.1.238) from 10.254.185.16 eth0: 56(84) bytes of data.
From company.com (10.254.185.16) icmp_seq=1 Destination Host Unreachable
From company.com (10.254.185.16) icmp_seq=2 Destination Host Unreachable

--- google.com ping statistics ---
2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1008ms
pipe 2


telnet -b 10.254.185.16 google.com 80
Trying 172.217.1.238...
Connected to google.com.
Escape character is '^]'.

ping works fine via eth1, which is the default route.

ping -I eth1  -c2 google.com
PING google.com (172.217.1.238) from 10.0.2.4 eth1: 56(84) bytes of data.
64 bytes from lax17s02-in-f14.1e100.net (172.217.1.238): icmp_seq=1 ttl=49 time=11.5 ms
64 bytes from lax17s02-in-f14.1e100.net (172.217.1.238): icmp_seq=2 ttl=49 time=11.3 ms

--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 11.310/11.446/11.582/0.136 ms

Details

default route is through eth1.

route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.0.2.1        0.0.0.0         UG    0      0        0 eth1
10.0.2.0        *               255.255.255.0   U     0      0        0 eth1
10.254.184.0    *               255.255.248.0   U     0      0        0 eth0
192.168.122.0   *               255.255.255.0   U     0      0        0 virbr0


ip route
default via 10.0.2.1 dev eth1
10.0.2.0/24 dev eth1  proto kernel  scope link  src 10.0.2.4
10.254.184.0/21 dev eth0  proto kernel  scope link  src 10.254.185.16
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1

eth0 is configured through another routing table

ip route show table eth0
default via 10.254.184.1 dev eth0
10.254.184.0/21 dev eth0  scope link  src 10.254.185.16


ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 08:00:27:6f:a1:e6
inet addr:10.254.185.16  Bcast:10.254.191.255     Mask:255.255.248.0
inet6 addr: fe80::a00:27ff:fe6f:a1e6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:2123 errors:0 dropped:0 overruns:0 frame:0
TX packets:1280 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:237141 (237.1 KB)  TX bytes:225214 (225.2 KB)


iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:bootps

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             192.168.122.0/24     ctstate RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24     anywhere
ACCEPT     all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootpc

IP Rule 

ip rule
0:  from all lookup local
32763:  from all to 10.246.240.0/20 lookup eth0
32764:  from 10.246.240.0/20 lookup eth0
32765:  from 10.246.242.68 lookup eth0
32766:  from all lookup main
32767:  from all lookup default

Traceroute output

traceroute -T r2d2.company.com
traceroute to r2d2.company.com (10.254.194.217), 30 hops max, 60 byte packets
1  nambi-ubuntu-dell-t5600.company.com (10.254.194.217)  13.181 ms  13.164 ms  13.142 ms


traceroute -I r2d2.company.com
traceroute to r2d2.company.com (10.246.20.141), 30 hops max, 60 byte packets
1  10.0.2.1 (10.0.2.1)  0.178 ms  0.139 ms  0.137 ms
2  * * *
3  te1-30-sjl1-2-cc01.company.com (10.246.100.81)  3.193 ms  3.174 ms  3.520 ms
4  te1-2-sjl2-2-cd02.companyy.com (10.246.100.62)  2.771 ms  2.853 ms  2.835 ms
5  * * *
6  * * *
7  * * *
8  * * *
9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
Nambi
  • 149
  • 1
  • 1
  • 5
  • Why does r2d2.company.com resolve to 2 different IPs? Which host are you unable to connect to? What is the command that you'd expect to succeed and with which output? I also don't understand why that traceroute to 10.254.194.217 would be successful with 1 hop. According to the other commands, that network is only reachable via gateway. Check `ip route get `. – al. Mar 17 '16 at 23:26
  • because it has two interfaces, eth0 and eth1. eth0 and eth1 are connected to different networks, that's why traceroute shows different hops. – Nambi Mar 18 '16 at 17:40

2 Answers2

0

Instead of simple ping, use traceroute -I. You can also use iptables to log your pings and your telnet packets with outgoing interfaces.

My recollections of bizarre configurations like yours are a bit hazy, but you will very probably find that even though you specify a source IP, that does not mean that your packets will leave through that interface! I'd say your ping packets do but your telnet packets don't, and something is PATting and reverse-routing them correctly. You say that eth0 has a different routing table, and you show the table, but I don't see the rule that sends packets to that table. Execute ip rule list to see those rules.

Law29
  • 3,557
  • 1
  • 16
  • 28
  • Added the output from "ip rule" command. – Nambi Mar 17 '16 at 21:06
  • Thanks for the suggestion to use "traceroute". Updated the thread with traceroute output. – Nambi Mar 17 '16 at 21:15
  • Looking at the ip rule output, I can see no reason why the routing table "eth0" should be used, so your results are not surprising. As for the traceroute, I was expecting the output of `traceroute -I -b eth0 google.com` and of `traceroute -I -s 10.254.185.16 google.com`, which will probably show you that your packets are leaving by eth1. What is the practical use of this setup? – Law29 Mar 17 '16 at 21:31
  • This is quite useful in separating traffic. btw, I tried "ssh -b remote-host" . Then verified by which ip address the remote host is accessed from by "w -i" command. Remote host reports eth0 or eth1 based on the -b option provided. So, the traffic does go through the different interfaces when routing is setup right. The only issue is "icmp" packets are not getting routed properly. – Nambi Mar 18 '16 at 16:18
  • The remote host you've given is google.com, or r2d2 which seems to have different IP addresses. Please note that when you force binding to a specific IP or interface, the source IP address of a packet does **not** mean that the packet actually exits your machine through the interface that has that IP. The exit interface is chosen according the routing table, chosen by the `ip rule`, and that does not show any non-local routing through eth0. – Law29 Mar 19 '16 at 08:47
0
  1. You are asking to ping -I eth0, so ping will try and send the packet using 10.254.185.16 as the source IP.
  2. Based on your ip rules, the system will use the main routing table and route the ICMP packet using the default gateway 10.0.2.1 to reach 172.217.1.238.

=> 10.0.2.1 cannot be reached from the source IP 10.254.185.16 (they are not in the same subnet), hence the unreachable ping response:

From company.com (10.254.185.16) icmp_seq=1 Destination Host Unreachable

To fix this, you could add the following ip rule to ensure proper table lookup:

ip rule add from 10.254.185.16/32 lookup eth0
Gohu
  • 121
  • 1
  • 1
  • 7