Persistent Volume for Jenkins on Kubernetes

Question

I'm trying to deploy a Jenkins image on a local Kubernetes cluster. The deployment is succesfull but I can't get the persistence data working. No errors are being thrown and new pods start up succesfully, the only problem is that it's not persistent.

Jenkins Dockerfile:

FROM jenkins/jenkins:lts

ENV JENKINS_USER admin
ENV JENKINS_PASS admin

# Skip initial setup
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false


COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/plugins.txt
USER root
RUN apt-get update \
  && apt-get install -qqy apt-transport-https ca-certificates curl gnupg2 software-properties-common 
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
RUN add-apt-repository \
  "deb [arch=amd64] https://download.docker.com/linux/debian \
  $(lsb_release -cs) \
  stable"
RUN apt-get update  -qq \
  && apt-get install docker-ce -y
RUN usermod -aG docker jenkins
RUN apt-get clean
RUN curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose
USER jenkins

Kubernetes Deployment file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
        - name: jenkins
          image: mikemanders/my-jenkins-image:1.1
          env:
            - name: JAVA_OPTS
              value: -Djenkins.install.runSetupWizard=false
          ports:
            - name: http-port
              containerPort: 8080
            - name: jnlp-port
              containerPort: 50000
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/lib/jenkins
              subPath: jenkins
      volumes:
        - name: jenkins-home
          persistentVolumeClaim:
            claimName: jenkins-pv-claim

Kubernetes persistent volume:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-pv
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 6Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/var/lib"

Persistent Volume Claim

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

I'm using Minikube for local development. And of course Kubectl.

Don't see what i'm doing wrong. All help is appreciated.

score 2 · Accepted Answer · answered May 04 '20 at 12:16

2

The regular dockerhub jenkins image uses the path /var/jenkins_home, not /var/lib/jenkins for persistent data. Therefore you should use that path to mount your persistent volume.

answered May 04 '20 at 12:16

Thomas

11,272
2
24
40

I tried your suggestion. The pod status is CrashLoopBackOff. When I look at the logs I get this message: `touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Permission denied Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions? `. I'm guessing the hostpath of the persistent volume is also not correct. – Mike Manders May 04 '20 at 12:53
1

I found a solution for this. `initContainers: - name: set-jenkins-home-permissions image: alpine:3.11 command: - chown - -R - 1000:1000 - /var/jenkins_home volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home` Not sure if this is a good way to solve this problem, but it fixes it for now so I can move on. But any better suggestion are always welcome. – Mike Manders May 04 '20 at 13:08
You can use a `securityContext` and set the `fsGroup` to `1000`, while not tested myself, it should give you the same result as your init container. More details: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/ – Thomas May 04 '20 at 14:14
@MikeManders, please, post your solution as an answer so others can be helped in the future if they face similar issue. – Mark Watney May 05 '20 at 09:46
yes @Thomas is correct you can see an example with `fsGroup: 1000` here https://medium.com/swlh/set-up-jenkins-in-a-kubernetes-cluster-96660c8d9ab – Britt Kelly Oct 05 '20 at 20:41

Persistent Volume for Jenkins on Kubernetes

1 Answers1