2

The overlay packets have to traverse a complex path in the kernel (involving multiple network stacks) before going out to the wire. When I trace the main kernel functions involved (using eBPF), the path of a packet looks something like this:

function             device
========             ======
sendto               veth0
sock_sendmsg         veth0
udp_sendmsg          veth0
udp_send_skb         veth0
ip_send_skb          veth0
ip_local_out         veth0
veth_xmit            veth0
do_softirq           (softirq handling)
net_rx_action        (softirq handling)
process_backlog      (softirq handling)
netif_receive_skb    vxlan0
br_forward           vxlan0
vxlan_xmit           vxlan0
udp_tunnel_xmit_one  eno1
ip_tunnel_xmit       eno1
ip_local_out         eno1
ixgbe_xmit_frame     eno1

Here, veth0 is the virtual interface in the overlay network, vxlan0 is the VxLAN overlay bridge, and eno1 is the physical interface.

So my question is that... since the sender doesn't know anything about the overlay network stuff (encapsulation, virtual network stacks, etc), does the sender's (running in the overlay network) sendto() return after sending the packet out of veth0? Or does it only return after the packet has left eno1?

I'm confused because it involves raising a softirq in the middle and softirqs are processed in a separate context/thread (but on the same core by default). I'm interested in this because I'm researching on improving the performance of overlay networks and I want to see if moving the softirq handling parts to a different cpu core can help.

munikarmanish
  • 352
  • 2
  • 3
  • 13

0 Answers0