3

We are trying to mount lustre filesystem inside running container, and have successfully done this via containers which are running in priviledged mode.

However for those containers which are running in non-privilidged mode, mounting lustre failed, even if all capabilites linux provides -- tens of capabilities -- were included!

Then

  1. what is difference between "priviledged: True" and "cap_add: all capabilites"?
  2. Why mounting lustre still fails when all capabilities were added to the container?

Mount Error enter image description here

Non-Privileged Mode Container:

version: "3"
services:
aiart:
cap_add:
  - AUDIT_CONTROL
  - AUDIT_READ
  - AUDIT_WRITE
  - BLOCK_SUSPEND
  - CHOWN
  - DAC_OVERRIDE
  - DAC_READ_SEARCH
  - FOWNER
  - FSETID
  - IPC_LOCK
  - IPC_OWNER
  - KILL
  - LEASE
  - LINUX_IMMUTABLE
  - MAC_ADMIN
  - MAC_OVERRIDE
  - MKNOD
  - NET_ADMIN
  - NET_BIND_SERVICE
  - NET_BROADCAST
  - NET_RAW
  - SETGID
  - SETFCAP
  - SETPCAP
  - SETUID
  - SYS_ADMIN
  - SYS_BOOT
  - SYS_CHROOT
  - SYS_MODULE
  - SYS_NICE
  - SYS_PACCT
  - SYS_PTRACE
  - SYS_RAWIO
  - SYS_RESOURCE
  - SYS_TIME
  - SYS_TTY_CONFIG
  - SYSLOG
  - WAKE_ALARM

image: test_lustre:1.1
#privileged: true
ports:
  - "12345:12345"
volumes:
  - /home/wallace/test-lustre/docker/lustre-client:/lustre/lustre-client
Wallace
  • 561
  • 2
  • 21
  • 54
  • Do you have the Lustre kernel modules loaded in the host before trying to mount inside the container? Any messages on the console? – LustreOne Feb 21 '21 at 18:34
  • Yes, mounting succeeded from host, and from within containers running in privileged mode. It fails only for containers running on normal mode. I have posted on picture containing the error msg. @LustreOne – Wallace Feb 22 '21 at 01:32
  • Pls add the exact command you are running for the non-privileged container call. It's hard to assess what "tens of capabilities" actually means. – Olesya Bolobova Feb 22 '21 at 04:14
  • added yaml file @OlesyaBolobova – Wallace Feb 22 '21 at 08:06

2 Answers2

1

Have you tried apparmor:unconfined?

version: "3"
services:
  aiart:
    cap_add:
    - SYS_ADMIN
    image: test_lustre:1.1
    security_opt:
    - apparmor:unconfined
    ports:
    - "12345:12345"
    volumes:
      - /home/wallace/test-lustre/docker/lustre-client:/lustre/lustre-client

If this works, then try to write a custom apparmor profile that fits your needs, because unconfined would be less secure I guess: https://docs.docker.com/engine/security/apparmor/

zsolt
  • 1,233
  • 8
  • 18
  • Your answer is right, apparmor:unconfined config works. But I still don't know the answers to my questions, and meanwhile niklas's posts answered. So I accepted his answer. – Wallace Feb 26 '21 at 01:21
1

The difference with --privileged and all-capabilities is, that --privileged argument removes all limitations enforced by cgroup controller and disables security enchantments while providing access for all devices. Privileged container truly becomes part of the host operating system, and has access even into AppArmor and SELinux configurations, which might not be applied, such as SELinux labels.

When --privileged flag is used, it does not enforce any extra security for underlying container, and kernel filesystem is not mounted as read-only into container. SECCOMP filtering is disabled as well. Still, you can't get more power than current namespace allows, for example if you are running rootless daemon.

Capabilities are way to adjust the power of root, but still some security enchantments are applied when container is executed.

One great blog post by Red Hat is available in here.

As pointed out in other answer, AppArmor is probably the issue in this case, and by using --security-opt apparmor:unconfined flag when running container, mounting might be possible. However, that should be used only temporally.

Niklas
  • 1,480
  • 4
  • 10