0

Recently I experienced an issue on running kubernetes slaves on a VM with calico ipip tunnel as the network connection protocol to interconnect the kubernetes nodes. The current symptom (with the confirmation of tcpdump) is that when a tcp traffic is initiated to the k8s slave node on VM, all the packets arrived at vmbr0 (bridge interface on VM host) with the size bigger than VM MTU are dropped and could not be found on the tap interface, which is connected to qemu VM. When the traffic is initiated without ipip header, the packets could be properly segmented on tcp level and forward to VM.

Note: The MTU on the link is 1500, and the bigger than VM MTU packet on vmbr0 is due to the GRO. One way to work around this is by disabling the GRO on the VM host physical interface, but it could introduce performance issue when serving high throughput traffics.

lordofire
  • 21
  • 3
  • Congratulations. You've discovered why it's important to have a consistent MTU on all devices on a segment. Your best bet is to raise MTU on the VMs, or lower it on the originator. – vidarlo May 08 '23 at 17:48
  • 1
    Hi @vidarlo , thanks for your comment. The reason I did not increase the MTU on VMs is that the increased packet payload comes from the GRO on the VM host driver, so logically the packet size could become really big (like 21786 when I watched the regular non-ipip traffic flowing through). As mentioned in the question, I could remove GRO so that all the packets coming from the VM host is strictly matching 1500 MTU, but the performance overhead will be too big a price to pay. – lordofire May 08 '23 at 20:02

0 Answers0