7

Linux has a feature to allow efficient capture of network packets by mmapping a shared buffer between the kernel and user. I'm trying to use this interface in a way that does not require root access (as I don't have it).

Often packet_mmap is used to look directly at all of the packets on the network, which would require root access. My application only requires the use of the standard linux UDP socket interface. I wish to use packet_mmap purely for efficiency - right now syscalls are eating over 50% of my CPU cycles.

Is there a way to configure packet_mmap such that it can be used from userspace?

charliehorse55
  • 1,940
  • 5
  • 24
  • 38
  • In Unix, the standard method of privilege elevation is the setuid bit. IMHO it would be wrong to use this kind of feature without requiring elevation. – Paulo Scardine Mar 01 '13 at 13:04
  • I am wondering if there was a way to restrict what I'm able to do with it so that I would be allowed to use it from userspace. – charliehorse55 Mar 01 '13 at 13:24

2 Answers2

4

While this doesn't truly answer the question (since it's specifically about packet_mmap), given your parameters:

  1. Receiving UDP packets
  2. Want to reduce syscalls, nothing else.
  3. Willing to use Linux-specific features, but no root user
  4. Features of packet_mmap not really needed or desired.

I would recommend you forget completely about packet_mmap and instead have a look at recvmmsg (note the spelling, not a typo).

recvmmsg does not require special privilegues, it is very intuitive (no obscure stuff, it works just like readv), and it lets you receive many packets in one call, greatly reducing syscall overhead.

Damon
  • 67,688
  • 20
  • 135
  • 185
  • Interesting, I'll take a look. I guess I will have to spawn a worker thread to call `recvmmsg` and place the result into a ringbuffer. (I need a ring buffer for efficient multithreading) – charliehorse55 Mar 01 '13 at 14:13
  • You can, or you could just [ensure that there is something to read](http://man7.org/linux/man-pages/man2/epoll_wait.2.html) first, so `recvmmsg` won't block. This is usually just as good as firing an extra thread (but less complex). – Damon Mar 01 '13 at 14:55
  • I thought about using that, but wouldn't it cause a lot of contention when multiple threads are trying to read at once? Additionally, one thread could end up receiving many packets at once and leaving none for the other threads. – charliehorse55 Mar 01 '13 at 21:24
  • Why would you receive on multiple threads in the first place? A single thread is easily sufficient to receive anything that comes over a gigabit or a 10G cable. You would normally have a single thread do nothing but receive, and optionally another thread (or another few) process the data, if you want to make use of parallelism. Having several threads receive data concurrently has no performance advantage, and may quite possibly be slower (lots of context switches). – Damon Mar 02 '13 at 16:38
  • That's why I was saying I'll have to create a thread to read the incoming packets into a ring buffer. – charliehorse55 Mar 03 '13 at 06:33
3

Looking at the Linux kernel Git repository, it appears that neither PF_INET sockets nor PF_INET6 sockets support memory-mapped access, so if by

My application only requires the use of the standard linux UDP socket interface. I wish to use packet_mmap purely for efficiency - right now syscalls are eating over 50% of my CPU cycles.

you mean you want to use memory-mapped access for a normal UDP or TCP socket, unfortunately, you can't. The same applies to raw IP sockets.

PF_PACKET sockets do support memory-mapped access, but they require elevated privileges, regardless of whether you're using memory-mapped access or not. They are not a replacement for PF_INET or PF_INET6 sockets; they are a mechanism for reading and writing link-layer packets, so if you want to run normal Internet applications atop them, good luck:

  1. you'll have to reimplement IP and whatever transport protocol you're using (UDP, TCP, etc.) yourself;
  2. you'll somehow have to keep the kernel's IP and transport protocol stack from processing those packets;

and you really don't want to try doing that.

(Note that by "elevated privileges" I don't necessarily mean "root privileges"; CAP_NET_RAW privileges should suffice. However, as I note, if you're trying to replace regular socket access, you don't want to use PF_PACKET sockets.)

  • You can use this interface to send standard UDP packets (SOCK_DGRAM) [see here](http://www.mjmwired.net/kernel/Documentation/networking/packet_mmap.txt) _"where mode is SOCK_RAW for the raw interface were link level information can be captured or SOCK_DGRAM for the cooked interface where link level information capture is not supported and a link level pseudo-header is provided by the kernel. "_ – charliehorse55 Mar 01 '13 at 20:55
  • 1
    You can, but you will have to construct the IP header yourself, and as for *receiving* standard UDP packets, if there's a process listening on the UDP port, it will get those packets even if you're using a `PF_PACKET` socket, and if there *isn't* a process listening on the UDP port, the kernel will send out an ICMP Port Unreachable message even if you're using a `PF_PACKET` socket. packet_mmap is *not* different from regular `PF_PACKET` sockets here, and neither are replacements for regular `PF_INET`/`PF_INET6` sockets. –  Mar 02 '13 at 00:17