6

Having this Dockerfile:

FROM fedora:30

ENV LANG C.UTF-8

RUN dnf upgrade -y \
    && dnf install -y \
        openssh-clients \
        openvpn \
        slirp4netns \
    && dnf clean all

CMD ["openvpn", "--config", "/vpn/ovpn.config", "--auth-user-pass", "/vpn/ovpn.auth"]

Building the image with:

podman build -t peque/vpn .

If I try to run it with (note $(pwd), where the VPN configuration and credentials are stored):

podman run -v $(pwd):/vpn:Z --cap-add=NET_ADMIN --device=/dev/net/tun -it peque/vpn

I get the following error:

ERROR: Cannot open TUN/TAP dev /dev/net/tun: Permission denied (errno=13)

Any ideas on how could I fix this? I would not mind changing the base image if that could help (i.e.: to Alpine or anything else as long as it allows me to use openvpn for the connection).

System information

Using Podman 1.4.4 (rootless) and Fedora 30 distribution with kernel 5.1.19.

/dev/net/tun permissions

Running the container with:

podman run -v $(pwd):/vpn:Z --cap-add=NET_ADMIN --device=/dev/net/tun -it peque/vpn

Then, from the container, I can:

# ls -l /dev/ | grep net
drwxr-xr-x. 2 root   root       60 Jul 23 07:31 net

I can also list /dev/net, but will get a "permission denied error":

# ls -l /dev/net
ls: cannot access '/dev/net/tun': Permission denied
total 0
-????????? ? ? ? ?            ? tun

Trying --privileged

If I try with --privileged:

podman run -v $(pwd):/vpn:Z --privileged --cap-add=NET_ADMIN --device=/dev/net/tun -it peque/vpn

Then instead of the permission-denied error (errno=13), I get a no-such-file-or-directory error (errno=2):

ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

I can effectively verify there is no /dev/net/ directory when using --privileged, even if I pass the --cap-add=NET_ADMIN --device=/dev/net/tun parameters.

Verbose log

This is the log I get when configuring the client with verb 3:

OpenVPN 2.4.7 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Feb 20 2019
library versions: OpenSSL 1.1.1c FIPS  28 May 2019, LZO 2.08
Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
TCP/UDP: Preserving recently used remote address: [AF_INET]xx.xx.xx.xx:1194
Socket Buffers: R=[212992->212992] S=[212992->212992]
UDP link local (bound): [AF_INET][undef]:0
UDP link remote: [AF_INET]xx.xx.xx.xx:1194
TLS: Initial packet from [AF_INET]xx.xx.xx.xx:1194, sid=3ebc16fc 8cb6d6b1
WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
VERIFY OK: depth=1, C=ES, ST=XXX, L=XXX, O=XXXXX, emailAddress=email@domain.com, CN=internal-ca
VERIFY KU OK
Validating certificate extended key usage
++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
VERIFY EKU OK
VERIFY OK: depth=0, C=ES, ST=XXX, L=XXX, O=XXXXX, emailAddress=email@domain.com, CN=ovpn.server.address
Control Channel: TLSv1.2, cipher TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
[ovpn.server.address] Peer Connection Initiated with [AF_INET]xx.xx.xx.xx:1194
SENT CONTROL [ovpn.server.address]: 'PUSH_REQUEST' (status=1)
PUSH: Received control message: 'PUSH_REPLY,route xx.xx.xx.xx 255.255.255.0,route xx.xx.xx.0 255.255.255.0,dhcp-option DOMAIN server.net,dhcp-option DNS xx.xx.xx.254,dhcp-option DNS xx.xx.xx.1,dhcp-option DNS xx.xx.xx.1,route-gateway xx.xx.xx.1,topology subnet,ping 10,ping-restart 60,ifconfig xx.xx.xx.24 255.255.255.0,peer-id 1'
OPTIONS IMPORT: timers and/or timeouts modified
OPTIONS IMPORT: --ifconfig/up options modified
OPTIONS IMPORT: route options modified
OPTIONS IMPORT: route-related options modified
OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
OPTIONS IMPORT: peer-id set
OPTIONS IMPORT: adjusting link_mtu to 1624
Outgoing Data Channel: Cipher 'AES-128-CBC' initialized with 128 bit key
Outgoing Data Channel: Using 160 bit message hash 'SHA1' for HMAC authentication
Incoming Data Channel: Cipher 'AES-128-CBC' initialized with 128 bit key
Incoming Data Channel: Using 160 bit message hash 'SHA1' for HMAC authentication
ROUTE_GATEWAY xx.xx.xx.xx/255.255.255.0 IFACE=tap0 HWADDR=0a:38:ba:e6:4b:5f
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)
Exiting due to fatal error

Error number may change depending on whether I run the command with --privileged or not.

Peque
  • 13,638
  • 11
  • 69
  • 105
  • Could not reproduce - the client container worked well with `CAP_NET_ADMIN` set and `/dev/net/tun` allowed. I used podman `1.4.4` (rootless), kernel `5.2.1`, keys and certs from the official openvpn samples, configurations - also from samples, but with some (probably) unneeded stuff removed. – Danila Kiver Jul 20 '19 at 17:44
  • Maybe it would make sense to add more specific information to the question - versions, minimal configurations for server/client, etc. – Danila Kiver Jul 20 '19 at 17:46
  • @DanilaKiver I appended more information to my question. More complete logs, information about how I see `/dev/net/tun` from the container and the results of trying `--privileged` (which did not help, or made it worse as `/dev/net/` did not exist then). Any ideas? – Peque Jul 23 '19 at 07:37
  • thanks, this update makes more sense now - Fedora has SELinux (checked on VM - in case of unprivileged container, SELinux blocks the access to `/dev/net/tun`). Also, It seems that absence of `/dev/net/tun` in case of privileged container is reproducible even out of Fedora and without SELinux - that's interesting. Going to dive a bit deeper. – Danila Kiver Jul 23 '19 at 08:50
  • @DanilaKiver You are right, if I `setenforce 0` from the host, then it works fine (i.e.: I have permissions to access `/dev/net/tun` from the container and I am able to connect to the VPN). – Peque Jul 23 '19 at 09:03
  • @DanilaKiver Do you have any idea on how could I workaround the permissions limitation without having to fully disable SELinux from the host? Maybe that is another question though... :innocent: – Peque Jul 23 '19 at 09:05
  • 1
    I think that the good way here is to extend existing SELinux policies to allow this specific container work with `tun_tap_device_t` - let me experiment for a while :) – Danila Kiver Jul 23 '19 at 09:33
  • Try with the `Dockerfile` I had in https://stackoverflow.com/questions/46472427/docker-vpn-ipsec-client-without-privileged-access/46617273#46617273, its a better option to use ubuntu instead of fedora – Tarun Lalwani Jul 24 '19 at 11:37

1 Answers1

8

It turns out that you are blocked by SELinux: after running the client container and trying to access /dev/net/tun inside it, you will get the following AVC denial in the audit log:

type=AVC msg=audit(1563869264.270:833): avc:  denied  { getattr } for  pid=11429 comm="ls" path="/dev/net/tun" dev="devtmpfs" ino=15236 scontext=system_u:system_r:container_t:s0:c502,c803 tcontext=system_u:object_r:tun_tap_device_t:s0 tclass=chr_file permissive=0

To allow your container configuring the tunnel while staying not fully privileged and with SELinux enforced, you need to customize SELinux policies a bit. However, I did not find an easy way to do this properly.

Luckily, there is a tool called udica, which can generate SELinux policies from container configurations. It does not provide the desired policy on its own and requires some manual intervention, so I will describe how I got the openvpn container working step-by-step.

First, install the required tools:

$ sudo dnf install policycoreutils-python-utils policycoreutils udica

Create the container with required privileges, then generate the policy for this container:

$ podman run -it --cap-add NET_ADMIN --device /dev/net/tun -v $PWD:/vpn:Z --name ovpn peque/vpn
$ podman inspect ovpn | sudo udica -j - ovpn_container

Policy ovpn_container created!

Please load these modules using: 
# semodule -i ovpn_container.cil /usr/share/udica/templates/base_container.cil

Restart the container with: "--security-opt label=type:ovpn_container.process" parameter

Here is the policy which was generated by udica:

$ cat ovpn_container.cil 
(block ovpn_container
    (blockinherit container)
    (allow process process ( capability ( chown dac_override fsetid fowner mknod net_raw setgid setuid setfcap setpcap net_bind_service sys_chroot kill audit_write net_admin ))) 

    (allow process default_t ( dir ( open read getattr lock search ioctl add_name remove_name write ))) 
    (allow process default_t ( file ( getattr read write append ioctl lock map open create  ))) 
    (allow process default_t ( sock_file ( getattr read write append open  ))) 
)

Let's try this policy (note the --security-opt option, which tells podman to run the container in newly created domain):

$ sudo semodule -i ovpn_container.cil /usr/share/udica/templates/base_container.cil
$ podman run -it --cap-add NET_ADMIN --device /dev/net/tun -v $PWD:/vpn:Z --security-opt label=type:ovpn_container.process peque/vpn
<...>
ERROR: Cannot open TUN/TAP dev /dev/net/tun: Permission denied (errno=13)

Ugh. Here is the problem: the policy generated by udica still does not know about specific requirements of our container, as they are not reflected in its configuration (well, probably, it is possible to infer that you want to allow operations on tun_tap_device_t based on the fact that you requested --device /dev/net/tun, but...). So, we need to customize the policy by extending it with few more statements.

Let's disable SELinux temporarily and run the container to collect the expected denials:

$ sudo setenforce 0
$ podman run -it --cap-add NET_ADMIN --device /dev/net/tun -v $PWD:/vpn:Z --security-opt label=type:ovpn_container.process peque/vpn

These are:

$ sudo grep denied /var/log/audit/audit.log
type=AVC msg=audit(1563889218.937:839): avc:  denied  { read write } for  pid=3272 comm="openvpn" name="tun" dev="devtmpfs" ino=15178 scontext=system_u:system_r:ovpn_container.process:s0:c138,c149 tcontext=system_u:object_r:tun_tap_device_t:s0 tclass=chr_file permissive=1
type=AVC msg=audit(1563889218.937:840): avc:  denied  { open } for  pid=3272 comm="openvpn" path="/dev/net/tun" dev="devtmpfs" ino=15178 scontext=system_u:system_r:ovpn_container.process:s0:c138,c149 tcontext=system_u:object_r:tun_tap_device_t:s0 tclass=chr_file permissive=1
type=AVC msg=audit(1563889218.937:841): avc:  denied  { ioctl } for  pid=3272 comm="openvpn" path="/dev/net/tun" dev="devtmpfs" ino=15178 ioctlcmd=0x54ca scontext=system_u:system_r:ovpn_container.process:s0:c138,c149 tcontext=system_u:object_r:tun_tap_device_t:s0 tclass=chr_file permissive=1
type=AVC msg=audit(1563889218.947:842): avc:  denied  { nlmsg_write } for  pid=3273 comm="ip" scontext=system_u:system_r:ovpn_container.process:s0:c138,c149 tcontext=system_u:system_r:ovpn_container.process:s0:c138,c149 tclass=netlink_route_socket permissive=1

Or more human-readable:

$ sudo grep denied /var/log/audit/audit.log | audit2allow


#============= ovpn_container.process ==============
allow ovpn_container.process self:netlink_route_socket nlmsg_write;
allow ovpn_container.process tun_tap_device_t:chr_file { ioctl open read write };

OK, let's modify the udica-generated policy by adding the advised allows to it (note, that here I manually translated the syntax to CIL):

(block ovpn_container
    (blockinherit container)
    (allow process process ( capability ( chown dac_override fsetid fowner mknod net_raw setgid setuid setfcap setpcap net_bind_service sys_chroot kill audit_write net_admin )))

    (allow process default_t ( dir ( open read getattr lock search ioctl add_name remove_name write )))
    (allow process default_t ( file ( getattr read write append ioctl lock map open create  )))
    (allow process default_t ( sock_file ( getattr read write append open  )))

    ; This is our new stuff.
    (allow process tun_tap_device_t ( chr_file ( ioctl open read write )))
    (allow process self ( netlink_route_socket ( nlmsg_write )))
)

Now we enable SELinux back, reload the module and check that the container works correctly when we specify our custom domain:

$ sudo setenforce 1
$ sudo semodule -r ovpn_container
$ sudo semodule -i ovpn_container.cil /usr/share/udica/templates/base_container.cil
$ podman run -it --cap-add NET_ADMIN --device /dev/net/tun -v $PWD:/vpn:Z --security-opt label=type:ovpn_container.process peque/vpn
<...>
Initialization Sequence Completed

Finally, check that other containers still have no these privileges:

$ podman run -it --cap-add NET_ADMIN --device /dev/net/tun -v $PWD:/vpn:Z peque/vpn
<...>
ERROR: Cannot open TUN/TAP dev /dev/net/tun: Permission denied (errno=13)

Yay! We stay with SELinux on, and allow the tunnel configuration only to our specific container.

Danila Kiver
  • 3,418
  • 1
  • 21
  • 31
  • Thanks for the detailed answer! I currently do not have access to the machine where I can try this. I will try it tomorrow, but it looks promising. ^^ – Peque Jul 23 '19 at 14:40
  • I am getting an `TCP/UDP: Socket bind failed on local address [AF_INET][undef]:0: Permission denied (errno=13)`. Audit log says `type=AVC msg=audit(1563968192.672:465): avc: denied { node_bind } for pid=19626 comm="openvpn" scontext=system_u:system_r:ovpn_container.process:s0:c582,c694 tcontext=system_u:object_r:node_t:s0 tclass=udp_socket permissive=0`. I tried adding `(allow process openvpn_port_t ( udp_socket ( name_bind )))` and `(allow process node_t ( udp_socket ( name_bind )))` to the policy without luck. Any ideas? Maybe I am using the wrong syntax. :-/ – Peque Jul 24 '19 at 11:39
  • 1
    I think it should be `(allow process node_t ( udp_socket ( node_bind )))` - the denied action is `node_bind`, not `name_bind`. – Danila Kiver Jul 24 '19 at 11:55
  • Thanks for your help. I still have not managed to make it work... :-( But your answer definitely solved my first problem. Now I can `ssh` to servers which do not require the VPN, but still cannot `ssh` to those which require the VPN. The `ssh` command gets "stucked". With no apparent error and no SELinux warning. `ping` is not working either, so maybe I still have some connection problems. – Peque Jul 24 '19 at 12:32
  • Actually, if I wait enough, I get the following error: `ssh: connect to host server.domain.com port 22: Connection refused \n kex_exchange_identification: Connection closed by remote host`. – Peque Jul 24 '19 at 12:37
  • Could you try to do this from the privileged container, launched on behalf of `root` (something like `sudo podman run --privileged ...`) to ensure that this is the networking issue rather than security-related one? – Danila Kiver Jul 24 '19 at 13:09
  • Yeah, it is the same, even if I run it with `sudo` and with `--privileged` option. – Peque Jul 24 '19 at 14:31
  • I think that in this case the question is in `podman` and/or `openvpn` networking rather than in security, and the next steps are to look at how the traffic actually flows (`tcpdump` both at the host and in the container to see, at which point it gets dropped), check the routes, etc. – Danila Kiver Jul 24 '19 at 14:50
  • Probably it is better to try solving this issue in a separate question (and linking it with current one), as this one gets pretty huge. – Danila Kiver Jul 24 '19 at 14:51
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/196937/discussion-between-peque-and-danila-kiver). – Peque Jul 24 '19 at 15:19
  • I asked a new question: https://stackoverflow.com/questions/57186289/cannot-ssh-from-container-with-openvpn – Peque Jul 24 '19 at 15:21
  • Commeting here to let others know that this worked perfectly for me, however i had to add getattr here: (allow process tun_tap_device_t ( chr_file ( getattr ioctl open read write ))) – nillenilsson Dec 18 '21 at 11:32