Multiple interface NAT routing

Question

I have multiple internet connections from different ISPs and I need to route certain traffic over a particular ISP link. I am using Debian for this.

The setup is as follows;

eth0 (IP address is 192.168.1.2 with gateway 192.168.1.1 (192.168.1.1 being a multiple ADSL WAN router with four different ADSL connections load balanced))

eth1 (IP address is 10.254.239.1. This is the local network and all workstations are getting assigned IP addresses through DHCP from this)

eth2 (IP address is (fake) 20.20.20.22 with gateway 20.20.20.21)

The goal is to route only particular traffic over eth2 and all others via eth0 which goes to the 4 port WAN router.

I've read through the lartc.org examples and many others but cannot get it to work...

What I have done is;

Added a route table to /etc/iproute2/rt_tables

201     fiber

Added the following to rc.local

ip route add 20.20.20.20 dev eth2 src 20.20.20.22 table fiber 
ip route add default via 20.20.20.21 table fiber
ip route add 20.20.20.20 dev eth2 src 20.20.20.22
ip rule add fwmark 2 table fiber

Modified iptables script to;

iptables -F
iptables -t nat -F
iptables -t mangle -F
iptables -X
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -m state --state NEW -i ! eth0 -j ACCEPT
iptables -A INPUT -m state --state NEW -i ! eth2 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i eth2 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
iptables -A FORWARD -i eth1 -o eth2 -j ACCEPT
iptables -t mangle -N fiber
iptables -t mangle -A fiber -j MARK --set-mark 2
iptables -t mangle -A fiber -j ACCEPT
#  ONLY ROUTE TRAFFIC GOING TO 1.2.3.x OVER THE FIBER LINK
iptables -t mangle -A PREROUTING -i eth1 -p tcp -d 1.2.3.4 --dport 80 -j fiber
iptables -t mangle -A PREROUTING -i eth1 -p tcp -d 1.2.3.5 --dport 80 -j fiber
iptables -t mangle -A PREROUTING -i eth1 -p tcp -d 1.2.3.4 --dport 443 -j fiber
iptables -t mangle -A PREROUTING -i eth1 -p tcp -d 1.2.3.5 --dport 443 -j fiber
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE
iptables -A FORWARD -i eth1 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth0 -j REJECT
iptables -A FORWARD -i eth2 -o eth2 -j REJECT

Enabled IP forwarding at the kernel level:

echo 1 > /proc/sys/net/ipv4/ip_forward

What am I missing or doing wrong? Any help much appreciated.

Answer 1

These rules probably don't do what you want. The first rule permits any new connection from eth1 and eth2, the second permits everything from eth0 and eth1. This appears to permit everything from any interface basically making your firewall pointless.

iptables -A INPUT -m state --state NEW -i ! eth0 -j ACCEPT
iptables -A INPUT -m state --state NEW -i ! eth2 -j ACCEPT

You may mean this instead.

iptables -A INPUT -m state --state NEW -i eth1 -j ACCEPT

Anyway, instead of adding things to rc.local I would probably update your /etc/network/interfaces to look like this. I am making some guesses about what you have for masks, so be sure to check and update. You probably should be adding all of the link routes to the fiber table.

auto eth0
iface eth0 inet static
  address 192.168.1.2
  netmask 255.255.255.0
  network 192.168.1.0
  broadcast 192.168.1.255
  gateway 192.168.1.1
  up ip route add table fiber scope link proto kernel dev eth0 192.168.1.0/24

auto eth1
iface eth1 inet static
  address 10.254.239.1
  netmask 255.255.255.0
  network 10.254.239.0
  broadcast 10.254.239.255
  up ip route add table fiber scope link proto kernel dev eth1 10.254.239.0/24

auto eth2
iface eth2 inet static
  address 20.20.20.22
  netmask 255.255.255.252
  network 20.20.20.20
  broadcast 20.20.20.23
  up ip route add table fiber scope link proto kernel dev eth2 20.20.20.20/30
  up ip route add default via 20.20.20.21 table fiber
  up ip rule add fwmark 2 table fiber