This answer will use policy routing where the fate of a packet is not determined only by its destination but also (and first) by other factors (here source IP address and/or incoming interface). Policy routing is not available with the Linux-obsolete route command but only with the newer API using ip rule and ip route (along ip link and ip address). The goal is to split a single view of all routes into multiple routing tables where each of them will offer a specific view of the routes intended to be used as if other not interesting parts didn't exist. Among other things, this allows to define multiple default routes to be used at the same time: one per routing table.
By default only this exists, (where the last rule's default table is empty):
# ip rule show
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
Here OP's case can be expressed with this kind of policy routing when handling a packet:
- packet from coglink or from coggw interfaces use routes for Cogeco dedicated routing table
- packet from ppp0 or from tekgw interfaces use routes for Teksavvy dedicated routing table
While one could just add rules to handle ppp0 and leave the main routing table handle coglink I'll still add rules for both, for the sake of symmetry.
Completing some blank spots and choosing arbitrary values (to provide commands without syntax error):
- coglink's local address is arbitrarily chosen as 67.193.56.92/21
- coglink's gateway is arbitrarily chosen as 67.193.56.1 instead of 67.193.x.x
- the routing table for Cogeco has the arbitrarily chosen value 7992
- ppp0's local address is arbitrarily chosen as 206.248.155.133
- assuming ppp0 is a layer 3 interface (even if over Ethernet, it's still Point-to-Point), so no gateway is needed for proper routing (else add in appropriate places
via 206.248.155.132).
- the routing table for Teksavvy has the arbitrarily chosen value 5645
Each routing table will include only the WAN to use rather than both, but must also include routes to networks that will make use of it. The duplication of these LAN routes is mandatory if SRPF is enabled, else these LAN routes can usually be omitted (as long as they are in the main routing table).
ip route add 67.193.56.0/21 dev coglink table 7992
ip route add default via 67.193.56.1 dev coglink table 7992
ip route add 192.168.1.0/24 dev coggw table 7992
ip rule add iif coglink lookup 7992
ip rule add iif coggw lookup 7992
ip route add default dev ppp0 table 5645
ip route add 10.0.0.0/8 dev tekgw table 5645
ip rule add iif ppp0 lookup 5645
ip rule add iif tekgw lookup 5645
This won't affect the traffic with the router as end node itself, which will continue to use only the main table (and it's default route with lower metric: coglink's) because:
- (always the case) incoming traffic to the router's own IP addresses is handled in the local routing table looked up first by the policy rule with preference 0. No further tables are looked up.
- outgoing traffic won't be selected by any of the added policy rules
So some corner cases will fail for the Fedora router only (when not actually routing), for example choosing ppp0's IP address as source with a destination on Internet will still select the route through coglink and won't work correctly. This is what happens when the router answers a ping on its ppp0 IP address.
So to address the title of the question, to have the router itself select the correct route when binding with the IP address on an interface and routing through the other interface, additional rules can be added selecting routing table from source address:
ip rule add from 67.193.56.92 lookup 7992
ip rule add from 206.248.155.133 lookup 5645
Even only knowing a possible range of addresses (eg: retrieved using the whois command) is good enough by using an additional selector iif lo with the special meaning from local system (rather than from lo interface). The two previous rules can be replaced with:
ip rule add from 67.193.48.0/20 iif lo lookup 7992
ip rule add from 206.248.128.0/18 iif lo lookup 5645
For other cases (when the socket is not bound initially), the main routing table's default route with lowest metric will choose the interface (currently coglink) and by default its IP address as source.
Notes and caveat:
the NAT rules provided by iptables or nftables are still needed
With iptables this could look like this now that the outgoing interface is always chosen correctly:
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o coglink -j MASQUERADE
iptables -t nat -A POSTROUTING -s 10.0.0.0/8 -o ppp0 -j MASQUERADE
or with nftables and kernel >= 5.5 ingress interface can be used in postrouting:
mynat.nft (to be used with nft -f mynat.nft):
table ip mynat
delete table ip mynat
table ip mynat {
chain mypost {
type nat hook postrouting priority srcnat; policy accept;
iif "coggw" oif "coglink" masquerade
iif "tekgw" oifname "ppp0" masquerade
}
}
Of course choosing simpler NAT rules would work.
should interfaces disappear and reappear, or simply go down and up, many settings previously done will have to be reapplied accordingly: routes will disappear and must be reapplied, and routing rules referencing interfaces might perhaps become stale in some cases.
So some integration with the tool(s) handling network at boot and later must be done to keep these routing tables and routing rules working.
If the tools have limitations, some policy routing rules could be changed to accomodate. For example with the current topology, iif tekgw is (almost but not exactly) equivalent to from 10.0.0.0/8.
even if coglink's address and default route's gateway change, these two pieces of information are still available... on coglink interface and on routes (and maybe also in the DHCP client's lease file if DHCP was used). It's just a case of reusing existing data for integration. Here's an example using JSON output and jq:
address:
ip -json -4 address show dev coglink primary | jq -r '.[].addr_info[] | "\(.local)/\(.prefixlen)"'
gateway:
ip -json -4 route show default dev coglink | jq -r '.[].gateway'
They could even be directly provided by hooks in the tools handling system networking, like NetworkManager.
