1

I am trying to detect packets with a VLAN tag. I have some PCAP files to containing VLAN tagged packets to test. A Wireshark screenshot of a sample packet:

enter image description here

After reading some tutorials, I wrote the following code:

#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/in.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>


#define bpf_printk(fmt, ...)                              \
({                                                        \
  char ____fmt[] = fmt;                                   \
  bpf_trace_printk(____fmt, sizeof(____fmt),              \
                   ##__VA_ARGS__);                        \
})

SEC("xdpvlan")
int myxdpprogram(struct xdp_md *ctx) {
  void *data = (void *)(long)ctx->data;
  void *data_end = (void *)(long)ctx->data_end;
  struct ethhdr *eth = data;

  if ((void*)eth + sizeof(*eth) <= data_end) {
    bpf_printk("h_proto is: 0x%x, ETH_P_8021Q is: 0x%x\n", bpf_ntohs(eth->h_proto), ETH_P_8021Q);
  }
  return XDP_PASS;
}

char _license[] SEC("license") = "GPL";

The output in /sys/kernel/debug/tracing/trace is like this:

bpf_trace_printk: h_proto is: 0x800, ETH_P_8021Q is: 0x8100

I expected:

bpf_trace_printk: h_proto is: 0x8100, ETH_P_8021Q is: 0x8100

I am using Fedora 34 to test, kernel version: 5.11.12-300.fc34.x86_64. Why the h_proto is not equal to 0x8100?

Update

I have two VMs, and I am using tcpreplay to send packets (PCAP file) from one VM to the other VM that has the eBPF program. VMs are connected through a host-only interface. I load the program using: ip link set dev ens37 xdpgeneric obj xdp_vlan_kern.o sec xdpvlan

a5hk
  • 7,532
  • 3
  • 26
  • 40
  • How did you load the BPF program? How did you feed it the pcap file(s)? – pchaigno May 21 '21 at 08:15
  • @pchaigno, I have updated the question with the info. Please see the updated question. – a5hk May 21 '21 at 08:20
  • 2
    I think the hook for generic XDP is too high in the stack, the VLAN header may have been decapsulated already. You should be able to see it if you could try your program on a set up with driver-mode XDP. – Qeole May 21 '21 at 08:48
  • 1
    @Qeole, Thanks, I used `xdpdrv` on some `veth` and that solved the problem. – a5hk May 21 '21 at 09:46

4 Answers4

1

[EDIT] Not sure this answer is correct, have a look at the comments for details.

Generic XDP, or SKB-mode XDP, is an XDP mode that was primarily added for experimenting with XDP (and to provide a model for future driver-based implementations). Given that it requires no support from the NIC driver, it is easier to use, but has lower performance than the other modes (driver/native XDP or XDP hardware offload).

One consequence of not having driver support is that the hook for generic XDP is necessarily higher in the networking stack when compared with native XDP. Generic XDP runs after the socket buffer (SKB) has been allocated. This means that some processing may already have occurred on your packets. In your case, the networking stack has already decapsulated the packets from their VXLAN headers, so you just observe regular IP packets.

Switching to driver-level XDP, providing your hardware (or virtual interface) uses a driver that supports it, should allow you to process your packets before they are sent to the kernel stack and before the VXLAN are removed.

Qeole
  • 8,284
  • 1
  • 24
  • 52
  • The behavior seems evironment dependent. I am testing on another VM, different hypervisor, `ububtu 20.04`, `kernel 5.4.0-42-generic`, and it works in generic XDP mode. – a5hk May 22 '21 at 06:23
  • I did another round of testing with my original test setup, this time using only one VM and using virtual interfaces instead of sending traffic from one VM to another. `xdpgeneric` works just fine this way. – a5hk May 22 '21 at 17:54
  • Hmm interesting, in that case it's maybe not related to XDP generic but to the way the packets are forwarded between two VMs? – Qeole May 23 '21 at 12:22
  • VMs are connected through a host-only interface. I haven't done any configuration on interfaces. If I send a packet on VM1 through the interface that is connected to the host-only interface, VM2 is able to see that packet on its interface that is connected to that host-only interface. – a5hk May 28 '21 at 06:48
1

I faced the same problem when running xdp in xdpdrv mode. In this tutorial I found notes about VLAN offloads on NIC interface:

Since XDP needs to see the VLAN headers as part of the packet headers, it is important to turn off VLAN hardware offload (which most hardware NICs support), since that will remove the VLAN tag from the packet header and instead communicate it out of band to the kernel via the packet hardware descriptor. The testenv script already disables VLAN offload when setting up the environment, but for reference, here is how to turn it off for other devices, using ethtool:

# Check current setting:
ethtool -k DEV | grep vlan-offload
# Disable for both RX and TX
ethtool --offload DEV rxvlan off txvlan off
# Same as:
# ethtool -K DEV rxvlan off txvlan off
Roman Sokolov
  • 47
  • 1
  • 10
  • VLAN offload was on, I turned it off on both VMs on the relevant interfaces, but that didn't solved the problem. – a5hk May 28 '21 at 06:41
0

I tried to use driver-mode as @Qeole, suggested. I created a pair of virtual interfaces because my NIC's driver didn't support driver specific hook.

ip link add dev veth1 type veth peer name veth2

The I loaded the program:

ip link set dev veth1 xdpdrv obj xdp_vlan_kern.o sec xdpvlan

And the replayed the PCAP file (on the same VM):

tcpreplay -i veth2 vlan.pcap

The output was as I expected:

bpf_trace_printk: h_proto is: 0x8100, ETH_P_8021Q is: 0x8100

a5hk
  • 7,532
  • 3
  • 26
  • 40
0

Roman Sokolov's answer is correct.
Disabling txvlan on the sending side fixed the error.

ip link add veth0 type veth peer name veth1
ip link add link veth0 name veth0.100 type vlan id 100

ip link set veth0 up
ip link set veth1 up
ip link set veth0.100 up

ip addr add 10.100.0.4/24 dev veth0.100

ethtool -K veth0 txvlan off

Attaching ebpf program on veth1 using SKB mode, injecting packets into veth0.100 (simply run arping would be enough), then I can get packets with vlan tags in my ebpf program.

However it didn't solve the problem when I only disable rxvlan on veth1. I didn't test this on a physical device yet, I'll try it later and modify this answer.

wkgcass
  • 117
  • 1
  • 5
  • I have tested this again, i.e. disabling `rxvlan` and `txvlan`. It didn't help in that particular setup that I have described in the question. However, it works just fine in my physical environment. – a5hk Aug 31 '21 at 16:05