0

I have a Linux based device with two NICs:

NIC1 eth0: 192.168.1.100/24; Gateway: 192.168.1.1

NIC2 eth1: 192.168.1.101/24; Gateway not set

This device communicates with an Application Server (IP: 10.1.1.1). The application logic associates the device with both its IPs therefore reaching the device with its primary IP or if this fails switching to secondary. This is supposed to provide redundancy at device NIC level. I cannot use bonding here as the server is expecting the device to have 2 IP addresses.

In case of no further configuration, if I unplug NIC1 the device can no longer communicates with the server, this is expected as NIC2 has no gateway associated.

If however I add a static route to the server network via NIC2 and the routing table would be 

Destination       Gateway         Genmask           Int
0.0.0.0           192.168.1.1     255.255.255.0     eth0
192.168.1.0       0.0.0.0         255.255.255.0     eth0
10.1.1.0          192.168.1.1     255.255.255.0     eth1

I'm expecting the following behaviour

  1. when both NICs are connected, server is reached via eth1 due to more specific route
  2. if only NIC1 is unplugged, server is reached via eth1 due to the static route
  3. if only NIC2 is unplugged, static route should disappear from routing table and server should be reached via eth0 due to default gateway

1 and 2 happen as expected. Instead, when NIC2 is unplugged, the server cannot be reached.

Why? If I put NIC1 and NIC2 on differnet subnets, point 3 doesn't fail i.e. whatever cable I unplug, the device is always reachable on the other NIC. What's the technical reason preventing this from happening when both NICs are set on the same subnet?

EDIT

I gave a look at ARP flux issue which seems to be more a problem at application level, but this is not the case here. ARP flux doesn't seem preventing hosts communication, so wouldn't explain why pinging NIC1 fails when NIC2 is unplugged.

kuma
  • 158
  • 9
  • check "ARP flux" on google or SF. That's only a part of the iceberg. If you don't want bonding, which would be working, you probably need policy routing etc. Any reason why bonding is not an option? – A.B Apr 16 '20 at 19:47
  • @A.B the device must have 2 IPs for being considered "redundant" by the application. So this would require putting the two NICs on two different subnets right? – kuma Apr 17 '20 at 11:27
  • @A.B bonding aside, I understand ARP flux could be a problem at application level, but this is not the case here. ARP flux doesn't seem preventing hosts communication, so wouldn't explain why point 3 fails – kuma Apr 17 '20 at 14:45
  • @A.B "unplugging the NIC doesn't remove the static route" can you please point to a source? If I put NIC1 and NIC2 on differnet subnets, point 3 doesn't fail i.e. whatever cable I unplug, the device is always reachable on the other NIC, how this can be? – kuma Apr 17 '20 at 16:05
  • `This is supposed to provide redundancy at device NIC level` It doesn't. This provides redundancy on the IP layer. If you want redundancy on the NIC level you need bonding, and in that case you only need one IP. – Gerald Schneider Apr 20 '20 at 12:02
  • @GeraldSchneider by that I meant that in case a NIC fails, there's the other one... and this actually works if the 2 NICs are on different subnets (tested). The point of my question is why this doesn't work when they're on the same subnet. And as said I can't use bonding here as the server is expecting the device to have 2 IP addresses (I edited the question for making it more clear) – kuma Apr 20 '20 at 13:29
  • There is no automatic mechanism to detect carrier loss and remove the static route. Certainly if the NIC were `ifdown`ed (or equivalent) the route will be removed. The ancient `netplug` and `ifplugd` could do so but I don't know of a modern replacement. – Mark Wagner Apr 21 '20 at 22:58
  • "There is no automatic mechanism to detect carrier loss and remove the static route" does that mean that the default linux behavior is to keep a static route even if I unplug the cable from the interface it is associated to? Then why my setup works (i.e. the device is always reachable when I unplug 1 cable) when eth1 is placed on a different subnet? – kuma Apr 29 '20 at 13:30

1 Answers1

0

Not enough reputation to comment, so I write this as an answer. I suggest you go the bonding or teaming way. The solution you try to give is very awkward. Setup a virtual bond/team interface containing the 2 physical NICs (active/active or active/passive) and then add 2 IPs on the virtual bond/team interface, in order to be considered redundant by the application.

Krackout
  • 1,575
  • 7
  • 20
  • thank you, but I've already pointed out that bonding is not an option here, as the server is expecting this device to have 2 IP addresses – kuma Apr 20 '20 at 13:24
  • But you will have two addresses, in one interface. It can be done on any interface, physical or virtual, no need to have 2 NICs for 2 IPs. – Krackout Apr 21 '20 at 14:43
  • "you will have two addresses in one interface" why? If I put the 2 NICs on 2 different subnets I will have redundancy at NIC level (if one fails, the device is still reachable) so 2 addresses on 2 interfaces. The point of my post is: why this doesn't seem to happen when the 2 NICs are in the same subnet? I mean, what's the technical reason preventing this from happening? – kuma Apr 21 '20 at 20:49