Rechability of different interfaces without having a route in a seperate routing table

Question

We're currently trying to route all packets from our guest vlan's (eth1.251) subnet trough a wireguard tunnel into the internet. To accomplish this we're using policy based routing with a rule to use the routing table 10 when the traffic is comming from our guest subnet:

32765:  from 10.251.0.0/16 lookup 10

In routing table 10 we're creating a default route towards our tunnel interface:

default dev wg1  scope link

All of our clients in our guest network are able to reache the internet through the wireguard tunnel which is expected, however the client's cannot reach the gateway for the guest network (10.251.0.1). TCPDump shows that the ICMP echo reply is routed back trough the wg1 interface to our tunnel endpoint, which is obviously not intended. A quick solution for this is to add the scope link route for the eth1.251 guest vlan interface to the routing table 10:

default dev wg1  scope link 
10.251.0.0/16 dev eth1.251  proto kernel  scope link 

Now the client can reach the router interface and it services.

---

On this router is another interface eth1 with the subnet 192.168.0.1/16. When we now delete our newly added 10.251.0.0/16 route in table 10 we cannot reach the routers interface on 10.251.0.1 anymore, however we're still able to reach the interface 192.168.0.1 from a client on the 10.251.0.0/16 subnet. The clients (e.g 192.168.0.2) behind the router in the 192.168.0.0/16 subnet aren't rechable from 10.251.0.0/16.

Main Question: Why can we reach the 192.168.0.1 interface ip on our router without an explicit routing table entry, but not the 10.251.0.1 interface ip from our clients in the guest 10.251.0.0/16 subnet?

Here's an overview of the network structure. I think this helps to unterstand our setup.

Answer 1

There's no general explanation, it's just the matter of following what happens in the routing settings.

10.251.0.1, is both a local router address and part of 10.251.0.0/16.

When receiving a packet to a local address, the router matches the local table using the very first ip rule with lowest preference: 0, before the rule for table 10, and matches. Remember that routing tables match on destinations, while usually the custom rules are configured to match on sources.

When the router replies, this time the local table doesn't match: 10.251.0.2 is not a local destination. Next rule is checked and matches from 10.251.0.0/16, looks up table 10 and packet goes through wg1.

For 192.168.0.1, receiving the packet is exactly like before with the local table. Now the answer doesn't match the additional rule, and the main routing table applies: it works as usual: a Linux system will answer from any of its IPs.

Again, for 192.168.0.2: it's not a local IP, so doesn't match the local table, but the query does match the added rule: packets are lost through wg1.

So copying a part of the main routing table to the extra table to avoid side effects helps.

Many of this can be tested with ip route get and the correct syntax, as long as there are no marks involved:

Without the additional entry in table 10:

# ip route get from 10.251.0.2 iif eth1.251 10.251.0.1
RTNETLINK answers: Invalid cross-device link
# sysctl -w net.ipv4.conf.eth1/251.rp_filter=2 #relax reverse path filtering
# ip route get from 10.251.0.2 iif eth1.251 10.251.0.1
local 10.251.0.1 from 10.251.0.2 dev lo table local 
    cache <local> iif eth1.251 

# ip route get from 10.251.0.2 iif eth1.251 192.168.0.1
local 192.168.0.1 from 10.251.0.2 dev lo table local 
    cache <local> iif eth1.251 
# ip route get from 10.251.0.2 iif eth1.251 192.168.0.2
192.168.0.2 from 10.251.0.2 dev wg1 table 10 
    cache iif eth1.251 

reply route:

# ip route get from 10.251.0.1 10.251.0.2
10.251.0.2 from 10.251.0.1 dev wg1 table 10 uid 0 
    cache 
# ip route get from 192.168.0.1 10.251.0.2
10.251.0.2 from 192.168.0.1 dev eth1.251 uid 0 
    cache 

When adding the entry in table 10:

# ip route get from 10.251.0.2 iif eth1.251 10.251.0.1 #even with strict reverse path filtering, since the reverse route is correct
local 10.251.0.1 from 10.251.0.2 dev lo table local 
    cache <local> iif eth1.251 
# ip route get from 10.251.0.1 10.251.0.2
10.251.0.2 from 10.251.0.1 dev eth1.251 table 10 uid 0 
    cache