Making an `ip rule` higher priority than `local`

Question

I'm trying to redirect a subsection of incoming traffic to a different destination using fwmarks.

The procedure

1) Mark matching incoming packets:

iptables -t mangle -A PREROUTING -i pppoe0 -p tcp -m tcp --dport 80 -j MARK --set-xmark 6

2) Add a rule to direct the marked packets to routing table "200".

ip rule add fwmark 6 table 200

3) Add a default route on routing table "200" to the new destination.

ip route all default via 192.168.33.2 table 200

The problem

iptables -L PREROUTING -t mangle -v shows packets matching the rule I made in step 1, however they never get forwarded to where I expect.

I think the problem is that the traffic is destined for an address which is considered local to the host, and the local ip rules are matching being the rule I added in step 2, e.g.

~# ip rule show
0:      from all lookup local   <--- taking priority...
32765:  from all fwmark 0x6 lookup 200   <--- ... over this.
32766:  from all lookup main
32767:  from all lookup default

Question: is there a way to make my rule take priority over the local rules?

Answer 1

You can specify the rule priority with the preference (or prio or pref etc.) keyword to override the default priority (which is usually "one below the lowest priority not equal to 0") that would be chosen for the rule.

You can also create a rule which duplicates an other rule but using a different priority, then delete the original rule that was at an other priority: that's the equivalent of changing the rule's priority.

Here's how you can do it here:

ip rule add preference 200 fwmark 0x6 lookup 200
ip rule add preference 300 lookup local
ip rule delete preference 0

This moves the rule looking up the local routing table from 0 to 300, leaving room for other rules to take precedence, like now the rule with preference 200. This order prevents any loss of connectivity (assuming routing table 200 doesn't create any loss of connectivity itself).

---

Example on SU where I made use of this in an answer to route (and tunnel) traffic normally intended for the local system without NAT nor fwmark:

Using Wireguard to essentially give a machine in local network a public address from VPS

---

Avoiding iptables for this is often the best way to get it working. This requires kernel >= 4.17:

Extends fib rule match support to include sport, dport and ip proto match (to complete the 5-tuple match support). Common use-cases of Policy based routing in the data center require 5-tuple match

Without iptables and marks this would look like:

ip rule add preference 200 iif pppoe0 ipproto tcp dport 80 lookup 200

Plus the reverse direction which depending on the problem could be an other interface or in some case the special words iif lo meaning locally initiated:

ip rule add preference 201 iif ... ipproto tcp sport 80 lookup 200

I can't give a complete answer because many factors are involved and it depends on the end goal, which isn't very clear here (you can take a look at my SU answer to see all what could go wrong and have to be altered, like ARP in some cases).