TL;DR
Extend the official image with the dynamic modules, and update the helm_release terraform resource to set the controller.image.registry, controller.image.image, controller.image.tag, controller.image.digest, and controller.image.digestChroot for your custom image along with a controller.config.main-snippet to load the dynamic module(s) in the main context.
This is similar to my previous answer for building modules using the official nginx image. You can extend the ingress-nginx/controller image, build the modules in one stage, extend the official image with the dynamic modules in another stage, and use the image in your helm_release. An example for extending the ingress-nginx/controller with the echo-nginx-module e.g.:
Docker
ARG INGRESS_NGINX_CONTROLLER_VERSION
FROM registry.k8s.io/ingress-nginx/controller:${INGRESS_NGINX_CONTROLLER_VERSION} as build
ARG INGRESS_NGINX_CONTROLLER_VERSION
ENV INGRESS_NGINX_CONTROLLER_VERSION=${INGRESS_NGINX_CONTROLLER_VERSION}
USER root
RUN apk add \
automake \
ca-certificates \
curl \
gcc \
g++ \
make \
pcre-dev \
zlib-dev
RUN NGINX_VERSION=$(nginx -V 2>&1 |sed -n -e 's/nginx version: //p' |cut -d'/' -f2); \
curl -L "http://nginx.org/download/nginx-${NGINX_VERSION}.tar.gz" | tar -C /tmp/nginx --strip-components=1 -xz
WORKDIR /src/echo-nginx-module
RUN curl -L https://github.com/openresty/echo-nginx-module/archive/refs/tags/v0.63.tar.gz | tar --strip-components=1 -xz
WORKDIR /tmp/nginx
RUN ./configure --with-compat --add-dynamic-module=/src/echo-nginx-module && \
make modules
FROM registry.k8s.io/ingress-nginx/controller:${INGRESS_NGINX_CONTROLLER_VERSION}
COPY --from=build /tmp/nginx/objs/ngx_http_echo_module.so /etc/nginx/modules/
... build and push the image e.g.: docker build --rm -t myrepo/ingress-nginx/controller:v1.5.1-echo --build-arg INGRESS_NGINX_CONTROLLER_VERSION=v1.5.1 . && docker push myrepo/ingress-nginx/controller:v1.5.1-echo
Terraform
Update the terraform helm_release resource to install the charts using the custom image and adding a main-snippet to set the load_module directive in the main context:
resource "helm_release" "ingress-nginx" {
name = "ingress-nginx"
namespace = "kube-system"
repository = "https://kubernetes.github.io/ingress-nginx"
chart = "ingress-nginx"
version = "3.34.0"
set {
name = "controller.image.registry"
value = "myrepo"
}
set {
name = "controller.image.image"
value = "ingress-nginx/controller"
}
set {
name = "controller.image.tag"
value = "v1.5.1-echo"
}
set {
name = "controller.image.digest"
value = "sha256:1b32b3e8c983ef4a32d87dead51fbbf2a2c085f1deff6aa27a212ca6beefcb72"
}
set {
name = "controller.image.digestChroot"
value = "sha256:f2e1146adeadac8eebb251284f45f8569beef9c6ec834ae1335d26617da6af2d"
}
set {
name = "controller.config.main-snippet"
value = <<EOF
load_module /etc/nginx/modules/ngx_http_echo_module.so;
EOF
}
}
The controller.image.digest is the image RepoDigest: docker inspect myrepo/ingress-nginx/controller:v1.5.1-echo --format '{{range .RepoDigests}}{{println .}}{{end}}' |cut -d'@' -f2
The controller.image.digestChroot is the Parent sha: docker inspect myrepo/ingress-nginx/controller:v1.5.1-echo --format {{.Parent}}
Test
- Create a
nginx pod: kubectl run nginx --image=nginx
- Expose the pod:
kubectl expose pod nginx --port 80 --target-port 80
- Create an ingress with a
server-snippet:
cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/server-snippet: |
location /hello {
echo "hello, world!";
}
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: echo.example.com
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: nginx
port:
number: 80
tls:
- hosts:
- echo.example.com
secretName: tls-echo
EOF
Using cert-manager for TLS certificates issuance and external-dns for DNS management.
- Test using
curl:
