2

I want to implement XDP_SHARED_UMEM: https://www.kernel.org/doc/html/latest/networking/af_xdp.html#xdp-shared-umem-bind-flag

The libbpf library function xsk_socket__create (https://github.com/libbpf/libbpf/blob/master/src/xsk.c) checks the xsk_umem->refcount value. In case it is greater than 1, the XDP_SHARED_UMEM option of a struct sockaddr_xdp is set.

So as far as I understand correctly, I "just" need to pass the original umem struct of the socket I want to share the umem with and the rest is done by libbpf.

The way I tried to do it was to let the first process copy its umem-struct into a shared-memory area where the second process could load it from. But because struct xsk_umem is defined in xsk.c it is hidden from the user and I am not able to do something like this:

memcpy(shdm_ptr, umem, sizeof(struct xsk_umem))

I don't know how they expect someone to use the shared umem feature?

binaryBigInt
  • 1,526
  • 2
  • 18
  • 44
  • Would the example available in [kernel samples](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/samples/bpf/xdpsock_user.c?h=v5.5#n279) be helpful here? – Qeole Mar 10 '20 at 10:16
  • As far as I understand, `umem->buffer` is just the pointer to the memory area but I don't get the other variables of `struct xsk_umem` such as `fd` or `refcount` which are necessary for `xsk_socket__create` to set `XDP_SHARED_UMEM` accordingly. I am probably wrong but I don't see how this function helps me – binaryBigInt Mar 10 '20 at 10:32
  • Right, sorry. I don't know then :/. Could be that libbpf just doesn't have those helpers yet. – Qeole Mar 10 '20 at 10:51
  • I see there's `xsk_umem__fd()` in libbpf, could you use it to get the file descriptor and pass it to other processes? (I have not tried, no idea if this is supposed to work) – Qeole Mar 10 '20 at 10:58
  • I could do that but again `xsk_socket__create` expects a `struct xsk_umem *umem`. In order to be able to pass the `fd` directly to `xsk_socket__create` I have to change the library (I think) – binaryBigInt Mar 10 '20 at 11:51

1 Answers1

3

So this was discussed on the xdp-newbies mailing list. Reporting here for the record.

It is not recommended to go with a multi-process setup as you are trying to do. Björn says:

Note that if you'd like to do a multi-process setup with shared umem, you: need to have a control process that manages the fill/completion rings, and synchronize between the processes, OR re-mmap the fill/completetion ring from the socket owning the umem in multiple processes and synchronize the access to them. Neither is pleasant.

Honestly, not a setup I'd recommend.

And he adds:

Just for completeness; To setup shared umem:

  1. create socket 0 and register the umem to this.
  2. mmap the fr/cr using socket 0
  3. create socket 1, 2, n and refer to socket 0 for the umem.

So, in a multiprocess solution step 3 would be done in separate processes, and step 2 depending on your application. You'd need to pass socket 0 to the other processes and share the umem memory from the process where socket 0 was created. This is pretty much a threaded solution, given all the shared state.

I advice not taking this path.

(Credits to Björn.)

Qeole
  • 8,284
  • 1
  • 24
  • 52