2

I am trying to run DPDK in a non-privileged docker container. While I can limit the container's privileges and specify the container as non-privileged, I still need to run a dpdk application (say testpmd) as root. I can also run the container as non-root and use sudo to start testpmd.

I was wondering if anyone is able to run dpdk (without the --no-huge option) as non-root user, inside a docker container. If so, are there certain privileges or permissions that need to be granted?

UPDATED:

I'm using DPDK 20.02. I think I've narrowed down the problem to a ulimit -l setting.

From testpmd: EAL: cannot set up DMA remapping, error 12 (Cannot allocate memory)

From Dmesg dmesg: [ 5697911.199003] vfio_pin_pages_remote: RLIMIT_MEMLOCK (65536) exceeded.

In response to Vipin:

Did you need to adjust the limits for the container? if so how? I am using helm to deploy the pods so I'm not sure if I can modify the docker run command, it looks like I would need to edit /etc/security/limits.conf on the host and redeploy.

Also, what did you use to give ownership of the fs? Doesn't having a non-privilieged container prevent you? For testing, I just sudo it, but ultimately I want to be able to drop SETUID/SETGID.

Jeremy Brown
  • 21
  • 3
  • with DPDK 18.11 and 19.11.3 LTS I am able to run in non-root on host. Insidie container I had to give ownership of hugepage and vfio device as non-root. Can you share DPDK version, OS, error details – Vipin Varghese Mar 11 '21 at 12:11
  • thanks for sharing the update on DPDK version and observation. it looks like there are 2 parts to your question. 1) what are the minimum settings to be done to run DPDK in non-privelleged mode and 2) what are the minimum settings to be done for docker to run in non provilleged mode? `If the question is related to DPDK` then you need to start by addressing `Error creating '/run/user/0/dpdk': Permission denied`, then `EAL: rte_service_init() failed`, then `EAL: FATAL: Cannot get hugepage informatio`. Can you share the logs of the error you are facing.? – Vipin Varghese Mar 15 '21 at 02:10
  • can you share an update on the problem you are facing? If you have been able to run with non privileged mode on host, you will only need to allow permission to file access in docker to right folders to run inside container. – Vipin Varghese Mar 19 '21 at 03:51
  • @JerenyBrown based on your edit `I just sudo it` and `I think I've narrowed down the problem to a ulimit -l setting`. does this mean `Docker run command is executed with non root` while `DPDK application is run with root privellege` inside docker image? If yes, can you please ensure first you have DPDK application on host running as non-root. Then run the same inside docker. That is what I did to make it work on 18.11 LTS – Vipin Varghese Mar 24 '21 at 01:44
  • @VipinVarghese well.. I actually use helm to deploy the pods as the federations are somewhat involved. The docker run is not exposed to me, but if yes the helm commands are run on the master node as root. I tried changing that limit before on the node in the /etc/security/limits.conf (and rebooted), but it didn't seem to be modified in the container as I expected. But that's a good idea, I'll see if dpdk runs as non-root on the node. – Jeremy Brown Mar 25 '21 at 19:21
  • thanks for the update. If you had full root privelelge and option to edit run command for containers/dockers, can you try ` --privileged -v /sys/bus/pci/drivers:/sys/bus/pci/drivers -v /sys/kernel/mm/hugepages:/sys/kernel/mm/hugepages -v /sys/devices/system/node:/sys/devices/system/node -v /dev:/dev `. I am not expert in helm or kubernetes so apologies in advance. – Vipin Varghese Mar 26 '21 at 02:39
  • related: [A: Running DPDK C program without root privileges](https://stackoverflow.com/q/65274122/427158) – maxschlepzig Aug 21 '21 at 15:37

1 Answers1

1

We can run DPDK on Host or inside docker with non root user.

To run DPDK as non-root user

  1. Create or choose the user without root privelleges
  2. set access to RUNTIME directory value as export XDG_RUNTIME_DIR=/tmp (since all users has access to tmp folder and on certain distros /var/run might not be accessible
  3. Mount the huge pages to similar folder mkdir -p /tmp/mnt/huge; mount -t hugetlbfs nodev /tmp/mnt/huge
  4. assign ownership to user to access the huge page as chown -R [non-root user]:[non-root user] /tmp/mnt/huge
  5. If access to devices are required check the same with either iommu or no-iommu drivers using lsmod | grep vfio
  6. change the ownership of the device chown -R [non-root user]:[non-root user] /dev/vfio/[device id]
  7. user DPDK rte_eal_init option --huge-dir o point to /tmp/mnt/huge
  8. Certian PMD might fail even after step 7, for those use option --legacy-mem this resolves the issue.

In order to run the DPDK application inside a docker, couple more things need to addressed

  1. Use DPDK either 19.11 LTS or greater (there are patches related to docker, namespace, memory limit)
  2. certain SE policy (Linux) does not allow sharing of huge Pages, so use option --in-memory to disable sharing of MMAP to huge pages (this should avoid most of the issues).

Note: assumption made

  1. there is only one single application to run on docker
  2. as mentioned in the comments --privileged -v /sys/bus/pci/drivers:/sys/bus/pci/drivers -v /sys/kernel/mm/hugepages:/sys/kernel/mm/hugepages -v /sys/devices/system/node:/sys/devices/system/node -v /dev:/dev are used to run DPDk in docker with sudo privelleges.
  3. I assume based on the question ulimit -c unlimited cannot be executed also.
  4. If there are multiple dockers running dpdk application always use --file-prefix to distinguish.
  5. I have not tried this with DPDK 21.02, 21.05, 21.08

[EDIT-1] the earlier question that got removed is running DPDK as non root

Vipin Varghese
  • 4,540
  • 2
  • 9
  • 25
  • Just noticed the older post with the same details were removed, hence using this question to define how to run DPDK application as non-root and How to run non-root in docker. – Vipin Varghese Sep 15 '21 at 03:51