Second interface not redirecting traffic

Question

I am trying to setup a redirector to nat https traffic destined to (eth0)172.31.14.66 -> 1.1.1.1 and http traffic destined to (eth1)172.31.14.48 -> 2.2.2.2.

Here are my iptables rules:

iptables -i eth0 -I INPUT -p tcp -m tcp --dport 443 -j ACCEPT
iptables -i eth0 -t nat -A PREROUTING -p tcp --dport 443 -j DNAT --to-destination 1.1.1.1:443

iptables -i eth1 -I INPUT -p tcp -m tcp --dport 443 -j ACCEPT
iptables -i eth1 -t nat -A PREROUTING -p tcp --dport 443 -j DNAT --to-destination 2.2.2.2:443

iptables -t nat -A POSTROUTING -j MASQUERADE
iptables -I FORWARD -j ACCEPT
iptables -P FORWARD ACCEPT
sysctl net.ipv4.ip_forward=1

The first redirection, (eth0)172.31.14.66 -> 1.1.1.1, works great but the second does not seem to send any traffic. Here is a tcpdump on the redirector when I try to curl 172.31.14.66 from a different host

root@ip-172-31-13-215:/home/ubuntu# tcpdump -i any "port 443"
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
16:09:06.348756 IP 99.244.96.100.59067 > 172.31.14.48.https: Flags [S], seq 3621687339, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 1986088032 ecr 0,sackOK,eol], length 0
16:09:07.352414 IP 99.244.96.100.59067 > 172.31.14.48.https: Flags [S], seq 3621687339, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 1986089032 ecr 0,sackOK,eol], length 0
16:09:08.358737 IP 99.244.96.100.59067 > 172.31.14.48.https: Flags [S], seq 3621687339, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 1986090033 ecr 0,sackOK,eol], length 0
16:09:09.350781 IP 99.244.96.100.59067 > 172.31.14.48.https: Flags [S], seq 3621687339, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 1986091034 ecr 0,sackOK,eol], length 0
16:09:10.355483 IP 99.244.96.100.59067 > 172.31.14.48.https: Flags [S], seq 3621687339, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 1986092034 ecr 0,sackOK,eol], length 0

I also notice that the iptables rule is receiving the curl traffic

root@ip-172-31-13-215:/home/ubuntu# iptables -vL -t nat -n
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
   22  1348 DNAT       tcp  --  eth1   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443 to:2.2.2.2:443
[...SNIP...]

Through investigating I found that changing my default route interface from eth0 to eth1 fixes my second route (eth1) but breaks the first one (eth0), does anyone know why the routing table would be causing this issue?

as a side question, I notice when I changed the default route I also could not ssh into my host using eth0 elastic IP and rather had to ssh using eth1 elastic IP, anyone know why this may occur aswell? Thanks!

EDIT: here are my network information:

ubuntu@ip-172-31-13-215:~$ ip route
default via 172.31.0.1 dev eth0 proto dhcp src 172.31.13.215 metric 100
172.31.0.0/20 dev eth1 proto kernel scope link src 172.31.14.48
172.31.0.0/20 dev eth0 proto kernel scope link src 172.31.14.66
172.31.0.1 dev eth0 proto dhcp scope link src 172.31.13.215 metric 100

root@ip-172-31-13-215:/home/ubuntu# ip route show table 1000
default via 172.31.0.1 dev eth1 proto static
172.31.5.60 dev eth1 proto static scope link
172.31.14.48 dev eth1 proto static scope link

root@ip-172-31-13-215:/home/ubuntu# ip rule
0:      from all lookup local
0:      from 172.31.14.48 lookup 1000
0:      from 172.31.5.60 lookup 1000
32766:  from all lookup main
32767:  from all lookup default

root@ip-172-31-13-215:/home/ubuntu# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc fq_codel state UP group default qlen 1000
    link/ether 06:15:21:49:f1:60 brd ff:ff:ff:ff:ff:ff
    inet 172.31.14.66/20 brd 172.31.15.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 172.31.13.215/20 brd 172.31.15.255 scope global secondary dynamic eth0
       valid_lft 3410sec preferred_lft 3410sec
    inet6 fe80::415:21ff:fe49:f160/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 06:bf:45:aa:7a:16 brd ff:ff:ff:ff:ff:ff
    inet 172.31.14.48/20 brd 172.31.15.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet 172.31.5.60/20 brd 172.31.15.255 scope global secondary eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::4bf:45ff:feaa:7a16/64 scope link
       valid_lft forever preferred_lft forever

if you are in the case described there: https://en.wikipedia.org/wiki/Multihoming#Multihoming_with_multiple_addresses , you need policy/source-based routing. It has few to do with iptables and much to do with routing. You should give your network configuration (eg: `ip -br link; ip -br address; ip route; ip rule`) . When you debug with tcpdump, don't use `-i any`. Use simultaneously one tcpdump per interface. — A.B, May 29 '21 at 18:26
thanks for the reply, I was curious, for my own understanding, why policy routing is needed. what exactly is denying traffic from redirected from eth1? is there some kind of strange nat misinterpretation between eth1 and eth0? — rooter, May 29 '21 at 18:42
thank you for your help. For brevity I used 2 IP addresses, but basically I have 4 elastic IPs, 2 are assigned to each interface. I would like for the elastic IPs on eth0 to redirect/nat requests to 1.1.1.1 and 2.2.2.2 for eth1. Do you mind explaining the masquerade issue a bit more, so since it is traversing with a different public source IP it never reaches those postrouting rules? also is their any better way to debug iptables issue or is combing over the config the best bet? — rooter, May 29 '21 at 20:41
Actually I might have made mistakes on my own in these comments. I'd rather not continue. It was just strange that initially you omitted to mention you were using ip rules at all. This kind of thing should always be written and explained — A.B, May 29 '21 at 21:00
those ip rules are automatically added by aws when you attach an additional interface to the ec2 — rooter, May 29 '21 at 22:34

Second interface not redirecting traffic

0 Answers0