0

Wondering what decides the outgoing interface. Here is my setup

Eno1 - 192.168.3 0.2/24, Gateway - 192.168.30.1
Eno2 - 192.168.50.2/24,  Gateway - 192.168.50.1

My routing table looks like this -

[root ~]# ip route
default
    nexthop via 192.168.30.2 dev eno1 weight 1
    nexthop via 192.168.50.2 dev eno2 weight 1
192.168.30.0/24 dev eno1 proto kernel scope link src 192.168.30.2 metric 100
192.168.50.0/24 dev eno2 proto kernel scope link src 192.168.50.2 metric 101

And

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
default         hostname        0.0.0.0         UG        0 0          0 eno1
192.168.30.0    0.0.0.0         255.255.255.0   U         0 0          0 eno1
192.168.50.0    0.0.0.0         255.255.255.0   U         0 0          0 eno2

After this config, when I try to ask linux kernel which interface it will use for a random destination...it plays a ping pong game between eno1 and eno2

[root@ ~]# ip route get 1.1.1.9
1.1.1.9 via 192.168.50.2 dev eno2 src 192.168.50.2
    cache
[root@ ~]# ip route get 1.1.1.10
1.1.1.10 via 192.168.30.2 dev eno1 src 192.168.30.2
    cache
[root@ ~]# ip route get 1.1.1.12
1.1.1.12 via 192.168.50.2 dev eno2 src 192.168.50.2
    cache
[root@-2 ~]# ip route get 1.1.1.13
1.1.1.13 via 192.168.30.2 dev eno1 src 192.168.30.2
    cache
[root@-2 ~]# ip route get 1.1.1.14
1.1.1.14 via 192.168.30.2 dev eno1 src 192.168.30.2

it feels like the outgoing interface is selected at random Can you please help understand what decide the outgoing interface?

Manny
  • 1

1 Answers1

0

This is called Equal-Cost Multipath (ECMP).

Linux is designed to distribute flows of packets using multiple paths. It will not happen for individual packets. The network stack calculates the hash over a subset of packet header fields. The calculated hash will be used for the next hop selection.

Note that balancing will not be perfect and there may be significant problems in the deployment in practice, as it is route-based, and routes are cached. This means often-used IPs will always use the same route.

casual1st
  • 1
  • 1
  • Thank you very much, this makes total sense. Can you help me understand if there is a way to failover? In other words, if a destination is not reachable from NIC1 and it receives a ICMP message host unreachable...how do I failover and make it use NIC2 and then give up instead of a single try/NIC? – Manny Nov 14 '22 at 23:34