0

I have two EC2 instances on AWS. I attach a second network interface to one of the EC2 instances and configure Redhat to use the new interface and IP.

The problem is that I can't ping the EC2 instance over the second NIC.

I was reading that asymmetric routing has to be prevented for this but I did not manage to this correctly. My steps were as follows.

1) Setup the new NIC because it does not get the new IPv4 automatically.

cd /etc/sysconfig/network-scripts/
cat ifcfg-eth0 > ifcfg-eth1

The eth1 config looks as follows.

BOOTPROTO=dhcp
DEVICE=eth1
HWADDR=02:d9:f6:0e:09:00
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
IPADDR=192.168.125.232

ifdown eth1
ifup eth1

ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 192.168.125.247  netmask 255.255.255.224  broadcast 192.168.125.255
        inet6 fe80::3d:5cff:fef4:f5a8  prefixlen 64  scopeid 0x20<link>
        ether 02:3d:5c:f4:f5:a8  txqueuelen 1000  (Ethernet)


eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 192.168.125.232  netmask 255.255.255.224  broadcast 192.168.125.255
        inet6 fe80::d9:f6ff:fe0e:900  prefixlen 64  scopeid 0x20<link>
        ether 02:d9:f6:0e:09:00  txqueuelen 1000  (Ethernet)

[...]

route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.125.225 0.0.0.0         UG    100    0        0 eth0
0.0.0.0         192.168.125.225 0.0.0.0         UG    101    0        0 eth1
192.168.125.0   0.0.0.0         255.255.255.0   U     100    0        0 eth1
192.168.125.224 0.0.0.0         255.255.255.224 U     100    0        0 eth0
192.168.125.224 0.0.0.0         255.255.255.224 U     101    0        0 eth1

2) Trying to setup asymmetric routing like this.

ip route add default via 192.168.125.225 dev eth0 tab 1
ip route add default via 192.168.125.225 dev eth1 tab 2

ip rule add from 192.168.125.247/24 tab 1

As soon as I run the above command I get disconnected from the instance and I can't reconnect via SSH. After this I just re-create the instance.

I never manage to run the last command, which I think is needed.

ip rule add from 192.168.125.232/24 tab 2

What am I missing? How do I set up the routing correctly for my setup?

EDIT #1: New try with new IP addresses won't work.

ip route add 192.168.125.224/27 dev eth0 table t1
ip route add 192.168.125.224/27 dev eth1 table t2
ip route add default via 192.168.125.225 dev eth0 table t1
ip route add default via 192.168.125.225 dev eth1 table t2

ip rule add from 192.168.125.243/27 table t1 priority 100
ip rule add from 192.168.125.232/27 table t2 priority 200
ip route flush cache

The workaround to disable source/destination check works but is not really something I want.

EDIT #2: After even more googling and pulling hair out of my head. Another try (not working :/).

ip route add default via 192.168.125.225 dev eth0 table t1
ip route add default via 192.168.125.225 dev eth1 table t2

ip rule add from 192.168.125.243/32 table t1 priority 100
ip rule add from 192.168.125.232/32 table t2 priority 200
ip route flush cache
Tony Stark
  • 382
  • 1
  • 5
  • 17
  • What are you trying to achieve with that? If you simply need multiple IP addresses in the same subnet for this host, IP aliases would do that. In other words, you would have one IP address on device `eth0` and another on device `eth0:1`. – telcoM Nov 24 '17 at 12:12
  • I want to have private floating IP in the end. This interface will float between both EC2 instances. – Tony Stark Nov 24 '17 at 12:17
  • I believe you are on the right track with what you are trying to do -- policy routes are required in order for this to work, although technically what you are doing is actually intended to *prevent* asymmetric routing, not *configure* asymmetric routing. It's not immediately clear what exactly isn't right about your configuration. – Michael - sqlbot Nov 24 '17 at 13:56
  • @Michael-sqlbot: Yes, you are correct. The wording was incorrect. I fixed that. What information would one need to tell what's wrong? – Tony Stark Nov 24 '17 at 14:11
  • Check out my previous answer to this https://serverfault.com/a/867154/162720 – strongjz Nov 25 '17 at 17:28

1 Answers1

0

The answer is to configure a route and a rule for both interfaces.

echo 100 t1 >> /etc/iproute2/rt_tables
echo 101 t2 >> /etc/iproute2/rt_tables

ip route add default via 192.168.125.225 dev eth0 table t1
ip rule add from 192.168.125.243/32 table t1 priority 100

ip rule add from 192.168.125.232/32 table t2 priority 200
ip route add default via 192.168.125.225 dev eth1 table t2

The ip commands can be added to /etc/rc.d/rc.local for persistent settings.

Tony Stark
  • 382
  • 1
  • 5
  • 17