Nexus on k3s on restart does not persist Users and data

Question

I have installed on K3S raspberry pi cluster nexus with the following setups for kubernetes learning purposes. First I created a StatefulSet:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nexus
  namespace: dev-ops
spec:
  serviceName: "nexus"
  replicas: 1
  selector:
    matchLabels:
      app: nexus-server
  template:
      metadata:
        labels:
          app: nexus-server
      spec:
        containers:
        - name: nexus
          image: klo2k/nexus3:latest
          env:
          - name: MAX_HEAP
            value: "800m"
          - name: MIN_HEAP
            value: "300m"
          resources:
            limits:
              memory: "4Gi"
              cpu: "1000m"
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - containerPort: 8081
          volumeMounts:
            - name: nexusstorage
              mountPath: /sonatype-work
        volumes:
            - name: nexusstorage
              persistentVolumeClaim:
                claimName: nexusstorage

Storage class

apiVersion: storage.k8s.io/v1   
kind: StorageClass   
metadata:   
  name: nexusstorage 
provisioner: driver.longhorn.io
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
  numberOfReplicas: "3"
  staleReplicaTimeout: "30"
  fsType: "ext4"
  diskSelector: "ssd"
  nodeSelector: "ssd"

pvc

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nexusstorage
  namespace: dev-ops
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: nexusstorage
  resources:
    requests:
      storage: 50Gi

Service

apiVersion: v1
kind: Service
metadata:
  name: nexus-server
  namespace: dev-ops
  annotations:
      prometheus.io/scrape: 'true'
      prometheus.io/path:   /
      prometheus.io/port:   '8081'
spec:
  selector: 
    app: nexus-server
  type: LoadBalancer
  ports:
    - port: 8081
      targetPort: 8081
      nodePort: 32000

this setup will spin up nexus, but if I restart the pod the data will not persist and I have to create all the setups and users from scratch.

What I'm missing in this case?

UPDATE

I got it working, nexus needs on mount permissions on directory. The working StatefulSet looks as it follow

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nexus
  namespace: dev-ops
spec:
  serviceName: "nexus"
  replicas: 1
  selector:
    matchLabels:
      app: nexus-server
  template:
      metadata:
        labels:
          app: nexus-server
      spec:
        securityContext:
          runAsUser: 200
          runAsGroup: 200
          fsGroup: 200
        containers:
        - name: nexus
          image: klo2k/nexus3:latest
          env:
          - name: MAX_HEAP
            value: "800m"
          - name: MIN_HEAP
            value: "300m"
          resources:
            limits:
              memory: "4Gi"
              cpu: "1000m"
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - containerPort: 8081
          volumeMounts:
            - name: nexus-storage
              mountPath: /nexus-data
        volumes:
            - name: nexus-storage
              persistentVolumeClaim:
                claimName: nexus-storage

important snippet to get it working

 securityContext:
          runAsUser: 200
          runAsGroup: 200
          fsGroup: 200

Answer 1

I'm not familiar with that image, although checking dockerhub, they mention using a Dockerfile similar to that of Sonatype. Then, I would change the mountpoint for your volume, to /nexus-data

This is the default path storing data (they set this env var, then declare a VOLUME). Which we can confirm, looking at the repository that most likely produced your arm-capable image

And following up on your last comment, let's try to also mount it in /opt/sonatype/sonatype-work/nexus3...

In your statefulset, change volumeMounts, to this:

      volumeMounts:
        - name: nexusstorage
          mountPath: /nexus-data
        - name: nexusstorage
          mountPath: /opt/sonatype/sonatype-work/nexus3
    volumes:
      - name: nexusstorage
        persistentVolumeClaim:
          claimName: nexusstorage

Although the second volumeMount entry should not be necessary, as far as I understand. Maybe something's wrong with your storage provider?

Are you sure your PVC is write-able? Reverting back to your initial configuration, enter your pod (kubectl exec -it) and try to write a file at the root of your PVC.