When running `stunnel` as non-root it errors with "setgroups: Operation not permitted"

Question

I need to authenticate to an OIDC server using a client cert (in addition to the normal OIDC client authentication)

To avoid modifying the oauth2-proxy code, I'm setting up a transparent encryption layer using stunnel inside a Docker image.
FWIW - I created a script with setuid bit to launch stunnel and that works, but I'm not sure it's the best solution.

I can start stunnel as root user, but when trying to run stunnel as non-root, it binds to the port, then immediately bails.

Error details -

oauth2-proxy-oauth2-proxy-1  | # # # ACTIVE CONFIG # # #
oauth2-proxy-oauth2-proxy-1  | foreground = yes
oauth2-proxy-oauth2-proxy-1  | setuid = 65532
oauth2-proxy-oauth2-proxy-1  | setgid = 65532
oauth2-proxy-oauth2-proxy-1  | socket = l:TCP_NODELAY=1
oauth2-proxy-oauth2-proxy-1  | socket = r:TCP_NODELAY=1
oauth2-proxy-oauth2-proxy-1  | cert = /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | client = yes
oauth2-proxy-oauth2-proxy-1  | TIMEOUTbusy = 600
oauth2-proxy-oauth2-proxy-1  | TIMEOUTclose = 600
oauth2-proxy-oauth2-proxy-1  | TIMEOUTconnect = 600
oauth2-proxy-oauth2-proxy-1  | TIMEOUTidle = 600
oauth2-proxy-oauth2-proxy-1  | [httpsconnect]
oauth2-proxy-oauth2-proxy-1  | accept = 0.0.0.0:8080
oauth2-proxy-oauth2-proxy-1  | connect = postman-echo.com:443
oauth2-proxy-oauth2-proxy-1  | # # # ACTIVE CONFIG # # #
oauth2-proxy-oauth2-proxy-1  | Starting stunnel...
oauth2-proxy-oauth2-proxy-1  | [ ] Initializing inetd mode configuration
oauth2-proxy-oauth2-proxy-1  | [ ] Clients allowed=512000
oauth2-proxy-oauth2-proxy-1  | [.] stunnel 5.66 on aarch64-alpine-linux-musl platform
oauth2-proxy-oauth2-proxy-1  | [.] Compiled with OpenSSL 3.0.5 5 Jul 2022
oauth2-proxy-oauth2-proxy-1  | [.] Running  with OpenSSL 3.0.7 1 Nov 2022
oauth2-proxy-oauth2-proxy-1  | [.] Threading:PTHREAD Sockets:POLL,IPv6 TLS:ENGINE,OCSP,PSK,SNI
oauth2-proxy-oauth2-proxy-1  | [ ] errno: (*__errno_location())
oauth2-proxy-oauth2-proxy-1  | [ ] Initializing inetd mode configuration
oauth2-proxy-oauth2-proxy-1  | [.] Reading configuration from file /etc/stunnel/stunnel.conf
oauth2-proxy-oauth2-proxy-1  | [.] UTF-8 byte order mark not detected
oauth2-proxy-oauth2-proxy-1  | [ ] No PRNG seeding was required
oauth2-proxy-oauth2-proxy-1  | [ ] Initializing service [httpsconnect]
oauth2-proxy-oauth2-proxy-1  | [ ] stunnel default security level set: 2
oauth2-proxy-oauth2-proxy-1  | [ ] Ciphers: HIGH:!aNULL:!SSLv2:!DH:!kDHEPSK
oauth2-proxy-oauth2-proxy-1  | [ ] TLSv1.3 ciphersuites: TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256
oauth2-proxy-oauth2-proxy-1  | [ ] TLS options: 0x2100000 (+0x0, -0x0)
oauth2-proxy-oauth2-proxy-1  | [ ] Session resumption enabled
oauth2-proxy-oauth2-proxy-1  | [ ] Loading certificate from file: /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | [ ] Certificate loaded from file: /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | [ ] Loading private key from file: /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | [ ] Private key loaded from file: /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | [ ] Private key check succeeded
oauth2-proxy-oauth2-proxy-1  | [:] Service [httpsconnect] needs authentication to prevent MITM attacks
oauth2-proxy-oauth2-proxy-1  | [ ] DH initialization skipped: client section
oauth2-proxy-oauth2-proxy-1  | [ ] ECDH initialization
oauth2-proxy-oauth2-proxy-1  | [ ] ECDH initialized with curves X25519:P-256:X448:P-521:P-384
oauth2-proxy-oauth2-proxy-1  | [.] Configuration successful
oauth2-proxy-oauth2-proxy-1  | [ ] Deallocating deployed section defaults
oauth2-proxy-oauth2-proxy-1  | [ ] Binding service [httpsconnect]
oauth2-proxy-oauth2-proxy-1  | [ ] Listening file descriptor created (FD=9)
oauth2-proxy-oauth2-proxy-1  | [ ] Setting accept socket options (FD=9)
oauth2-proxy-oauth2-proxy-1  | [ ] Option SO_REUSEADDR set on accept socket
oauth2-proxy-oauth2-proxy-1  | [ ] Service [httpsconnect] (FD=9) bound to 0.0.0.0:8080
oauth2-proxy-oauth2-proxy-1  | [!] setgroups: Operation not permitted (1)
oauth2-proxy-oauth2-proxy-1  | [ ] Unbinding service [httpsconnect]
oauth2-proxy-oauth2-proxy-1  | [ ] Service [httpsconnect] closed (FD=9)
oauth2-proxy-oauth2-proxy-1  | [ ] Service [httpsconnect] closed

Dockerfile

FROM quay.io/oauth2-proxy/oauth2-proxy:v7.3.0 as builder

FROM alpine:3
COPY --from=builder /etc/nsswitch.conf /etc/nsswitch.conf
COPY --from=builder /bin/oauth2-proxy /bin/oauth2-proxy
COPY --from=builder /etc/ssl/private/jwt_signing_key.pem /etc/ssl/private/jwt_signing_key.pem


ARG         STUNNEL_VERSION=${STUNNEL_VERSION:-5.66-r0}
ARG         LIBRESSL_VERSION=${LIBRESSL_VERSION:-3.6.1-r0}
ARG         APP_USER=${APP_USER:-65532}

RUN         apk update && apk add --no-cache openssl stunnel=${STUNNEL_VERSION} libressl==${LIBRESSL_VERSION}

ENV         ACCEPT_IP=0.0.0.0 \
            ACCEPT_PORT=8080 \
            SERVICE=httpsconnect \
            DESTINATION_PORT=443 \
            DESTINATION_HOST=0.0.0.0 \
            CLIENT=yes \
            STUNNEL_VERSION=${STUNNEL_VERSION} \
            APP_USER=${APP_USER}


COPY --chown=${APP_USER}:${APP_USER} docker-entrypoint.sh stunnel.sh /

RUN rm /etc/stunnel/stunnel.conf && \
    chown root:root /stunnel.sh && \
    chmod ug+s /stunnel.sh && \
    chmod +x /stunnel.sh && \
    chmod +x /docker-entrypoint.sh && \
    mkdir -p /var/log/stunnel && \
    touch /var/log/stunnel/stunnel.log && \
    chown -R ${APP_USER}:${APP_USER} /var/log/stunnel && \
    chown ${APP_USER}:${APP_USER} /etc/stunnel 

# UID/GID 65532 is also known as nonroot user in distroless image
USER ${APP_USER}:${APP_USER}

ENTRYPOINT [ "./docker-entrypoint.sh" ]

Off-topic since this is not a programming question. But the culprit are your setuid and setgid options, which can only work if run as root. See [the documented example config](https://www.stunnel.org/config_unix.html) for more about these options. — Steffen Ullrich, Dec 21 '22 at 00:56
Good point Steffen - this would have been better on serverfault.com. fwiw, the docs mention, but don't say it's required. "It is recommended to drop root privileges if stunnel is started by root". Although this is specifically in a Docker config, and there are many stack overflow questions about Docker configs, so the guidelines may be vague here. =\ — Jeremy, Dec 21 '22 at 01:03
It is recommended for root to drop privileges. Other users cannot drop the privileges since they don't have these in the first place. Changing the uid/gid only works for root. — Steffen Ullrich, Dec 21 '22 at 05:12

score 0 · Answer 1 · answered Dec 21 '22 at 18:08

It's possible to run stunnel as non-root.
I've modified the Dockerfile and entrypoint script to start the process as a defined user, update file/folder ownership to the provided user, and skipped the setid/setgid parameters for stunnel so that it doesn't try to change the process owner.
Also, I needed to update the PID location in the stunnel config.

Dockerfile -

FROM quay.io/oauth2-proxy/oauth2-proxy:v7.3.0 as builder

FROM alpine:3
COPY --from=builder /etc/nsswitch.conf /etc/nsswitch.conf
COPY --from=builder /bin/oauth2-proxy /bin/oauth2-proxy
COPY --from=builder /etc/ssl/private/jwt_signing_key.pem /etc/ssl/private/jwt_signing_key.pem


ARG         STUNNEL_VERSION=${STUNNEL_VERSION:-5.66-r0}
ARG         LIBRESSL_VERSION=${LIBRESSL_VERSION:-3.6.1-r0}
# UID/GID 65532 is also known as nonroot user in distroless image
ARG         APP_USER=${APP_USER:-65532}

RUN         apk update && apk add --no-cache openssl stunnel=${STUNNEL_VERSION} libressl==${LIBRESSL_VERSION}

ENV         ACCEPT_IP=0.0.0.0 \
            ACCEPT_PORT=8080 \
            SERVICE=httpsconnect \
            DESTINATION_PORT=443 \
            DESTINATION_HOST=0.0.0.0 \
            CLIENT=yes \
            STUNNEL_VERSION=${STUNNEL_VERSION} \
            APP_USER=${APP_USER}

COPY --chown=${APP_USER}:${APP_USER} docker-entrypoint.sh /

RUN rm /etc/stunnel/stunnel.conf && \
    chmod +x /docker-entrypoint.sh && \
    mkdir -p /var/log/stunnel && \
    chown ${APP_USER}:${APP_USER} /etc/stunnel 

USER ${APP_USER}:${APP_USER}

ENTRYPOINT [ "./docker-entrypoint.sh" ]

Entrypoint (adapted from another example)

#!/bin/sh

to_file() {

    TEXT="${1}"
    FILE="${2}"
    ECHO="$(command -v echo)"

    ${ECHO} "${TEXT}" >> "${FILE}"
}

cd /etc/stunnel

if [ -f stunnel.conf ]
then
    rm -f stunnel.conf
fi

to_file "
foreground = yes
pid = /etc/stunnel/stunnel.pid
debug = info
output = /etc/stunnel/stunnel.log
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
cert = /etc/stunnel/stunnel.pem
client = ${CLIENT:-no}" "stunnel.conf"

for DOM in $(echo $SNI | sed "s/,/ /g")
do
    to_file "SNI = ${DOM:-}" "stunnel.conf"
done

to_file "TIMEOUTbusy = 600
TIMEOUTclose = 600
TIMEOUTconnect = 600
TIMEOUTidle = 600
[${SERVICE}]
accept = ${ACCEPT_IP}:${ACCEPT_PORT}
connect = ${DESTINATION_HOST}:${DESTINATION_PORT}" "stunnel.conf"

if ! [ -f stunnel.pem ]
then
    openssl req -x509 -nodes -newkey rsa:2048 -days 3650 -subj '/CN=stunnel' \
                -keyout stunnel.pem -out stunnel.pem
    chmod 600 stunnel.pem
fi

echo "# # # ACTIVE CONFIG # # #"
cat "stunnel.conf"
echo "# # # ACTIVE CONFIG # # #"

echo "Starting stunnel..."
exec stunnel &
echo "Starting oauth2-proxy..."
exec /bin/oauth2-proxy "$@"

When running `stunnel` as non-root it errors with "setgroups: Operation not permitted"

1 Answers1