0

I'm having problems configuring iptables rules on a double wan setup. The OS is Ubuntu 12.04 LTS.

I have 2 Wan connections, both with static IP. Wan-1 is on eth2, Wan-2 is on eth4. I want both of them to forward to the same internal servers.

Wan-1 rules are working, and I can access all internal servers without problem, but I can't access servers when using Wan-2.

Wan-1: (not the real IP addresses)

IP: 17.13.12.90
CIDR: 17.13.12.90/29
Netmask: 255.255.255.248
Gateway: 17.13.12.89

Wan-2: (not the real IP addresses)

IP: 17.13.12.174
CIDR: 17.13.12.174/29
Netmask: 255.255.255.248
Gateway: 17.13.12.169

Wan-1 (eth2) is the current default gateway, everything is working here. I can access internal servers from the internet. No problems. I can ping it from the outside and connect via SSH.

Wan-2 (eth4) is the new wan connection I'm setting up. I can ping it from the outside and connect via SSH to it. But I can't connect to internal servers from this IP.

Local network is eth0.

My /etc/iproute2/rt_tables file looks like this:

#
# reserved values
#
255 local
254 main
253 default
0   unspec
#
# local
#
10  isp1
20  isp2

This are the ip rules I've setup:

ip route add 17.13.12.88 dev eth2 src 17.13.12.90 table isp1
ip route add default via 177.135.127.89 table isp1
ip route add 17.22.17.168 dev eth4 src 17.22.17.174 table isp2
ip route add default via 17.22.17.169 table isp2

ip route add 17.13.12.88 dev eth2 src 17.13.12.90
ip route add 17.22.17.168 dev eth4 src 17.22.17.174

ip route add default via 17.13.12.89

ip rule add from 17.13.12.90 table isp1
ip rule add from 17.22.17.174 table isp2
ip rule add fwmark 0x1 table isp1
ip rule add fwmark 0x2 table isp2

This is the iptables rules, concerning the mangle table:

$IPT -t mangle -A PREROUTING -p tcp -j CONNMARK --restore-mark

# Input rules
$IPT -t mangle -A INPUT -i $IF_ETH2 -p tcp -j MARK --set-mark 0xa
$IPT -t mangle -A INPUT -i $IF_ETH4 -p tcp -j MARK --set-mark 0xb

$IPT -t mangle -A PREROUTING -i $IF_ETH0 -m mark --mark 0xa -p tcp -j MARK --set-mark 0x1
$IPT -t mangle -A PREROUTING -i $IF_ETH0 -m mark --mark 0xb -p tcp -j MARK --set-mark 0x2

# output rules
$IPT -t mangle -A OUTPUT -m mark --mark 0xa -p tcp -j MARK --set-mark 0x1
$IPT -t mangle -A OUTPUT -m mark --mark 0xb -p tcp -j MARK --set-mark 0x2

Could someone please point me/help me out why aren't connexions coming from wan-2 being forward to the internal servers? The rules seem ok, but maybe I'm missing something.

Edmar
  • 1
  • 1
  • 2

2 Answers2

0

At the moment I'm using the following settings. Let's assume 192.168.127.10 it's our internal server and xxx.xxx.xxx.xxx/yyy.yyy.yyy.yyy ip addresses of wan1/wan2

# iptables -t mangle -I PREROUTING -s 192.168.127.10 -m conntrack --ctorigdst xxx.xxx.xxx.xxx -j MARK --set-mark 0x3e8
# iptables -t mangle -I PREROUTING -s 192.168.127.10 -m conntrack --ctorigdst yyy.yyy.yyy.yyy -j MARK --set-mark 0x7d0

# ip ru add fwmark 0x3e8 lookup ISP1 prio 1000
# ip ru add fwmark 0x7d0 lookup ISP2 prio 2000

Setup default policy base routing

# cat /root/routing/set_default_routing.sh
#!/bin/sh

IP1='xxx.xxx.xxx.xxx'
IF1='eth2'
P1='xxx.xxx.xxx.1'
P1_NET='xxx.xxx.xxx.0/29'

IP2='yyy.yyy.yyy.yyy'
IF2='eth4'
P2='yyy.yyy.yyy.25'
P2_NET='yyy.yyy.yyy.0/29'

/sbin/ip route add $P1_NET dev $IF1 src $IP1 table ISP1
/sbin/ip route add default via $P1 table ISP1

/sbin/ip route add $P2_NET dev $IF2 src $IP2 table ISP2
/sbin/ip route add default via $P2 table ISP2

/sbin/ip rule add from $IP1 table ISP1
/sbin/ip rule add from $IP2 table ISP2

/sbin/ip route add default via $P1
/sbin/ip route flush cache

P.S. don't forget to disable reverse path filter

# echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter

Add the following line to the /etc/sysctl.conf 

net.ipv4.conf.default.rp_filter = 0
ALex_hha
  • 7,193
  • 1
  • 25
  • 40
  • Hello Alex, thank you for your input. My routing tables are identical to yours. I've flushed all my rules and even used your script, to search for potencial errors I might have made. I've cleared my iptables mangle rules and added yours, but I still have the same problem... if the connection comes in from wan-1 all is OK, but with wan-2 it doesn't work. – Edmar Apr 07 '15 at 22:00
  • Add output of the ip ru sh/ip ro sh/ip ro sh table ISP1/ip ro sh table ISP2 to the question – ALex_hha Apr 08 '15 at 00:29
0

I've solved the problem. It was related to the reverse path filter. I had put this in my script:

echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter

But when I was troubleshooting the problem I realized that this value was 1 in all the interfaces (except default):

$ cat /proc/sys/net/ipv4/conf/*/rp_filter 
1
0
1
1
1
1
1
1
1
1

I've set the values to zero, and changed the iptables rules, according to this article:

http://nerdboys.com/2006/05/05/conning-the-mark-multiwan-connections-using-iptables-mark-connmark-and-iproute2/

This is my iptables script now (part of it):

# Disabling rp_filter   
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth2/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth4/rp_filter

# Mangle rules
$IPT -t mangle -A PREROUTING -p tcp -j CONNMARK --restore-mark
$IPT -t mangle -A PREROUTING -p tcp -i $IF_ETH2 -j MARK --set-mark 0x1
$IPT -t mangle -A PREROUTING -p tcp --match mark --mark 0x2 -j ACCEPT
$IPT -t mangle -A PREROUTING -p tcp -i $IF_ETH4 -j MARK --set-mark 0x2
$IPT -t mangle -A PREROUTING -p tcp -j CONNMARK --save-mark
Edmar
  • 1
  • 1
  • 2