How to install rabbitmq plugin on kubernetes?

Question

I have a Kubernetes environment with a rabbitmq servirve who deploys 2 pods of rabbitmq.

I need to install a plugin on rabbitmq, (Delayed Message Plugin) but I don't like the "manual" way, so if the pod is deleted, I have to install the plugin again.

I want to know which is the recommended way of achieving this.

FYI: the manual way is to copy a file into the plugins folder, and then launch the following command:

rabbitmq-plugins enable rabbitmq_delayed_message_exchange

Answer 1

You should mount the configuration for RabbitMQ from a config map.

For example:

The ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: rabbitmq-config
  namespace: rabbitmq
data:
  enabled_plugins: |
      [rabbitmq_management,rabbitmq_peer_discovery_k8s].
  rabbitmq.conf: |
      ...
  definitions.json: |
      ...

And then in your Deployment or StatefulSet:

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: rabbitmq
  namespace: rabbitmq
spec:
  replicas: 3
  ...
  template:
    ...
    spec:
      containers:
      - image: rabbitmq:3.7.4-management-alpine
        imagePullPolicy: IfNotPresent
        name: rabbitmq
        volumeMounts:
        - name: config-volume
          mountPath: /etc/rabbitmq
        ...
      volumes:
        - name: config-volume
          configMap:
            name: rabbitmq-config
            items:
            - key: rabbitmq.conf
              path: rabbitmq.conf
            - key: enabled_plugins
              path: enabled_plugins
            - key: definitions.json
              path: definitions.json
       ...

There are several ways to install the plugin in the first place. One is to base off of the image you are currently using, add the plugin, and use the new image instead. Alternatively you could utilize Kubernetes life cycle hooks to download the file pre start. Here is an example of postStart

Answer 2

I've ended mounting a persistent volumen to a shared hard driven, and using the life cycle hooks to copy the file to the correct path

  lifecycle:
    postStart:
      exec:
        command: ['sh', '-c', 'cp /data/rabbitmq_delayed_message_exchange-20171201-3.7.x.ez /opt/rabbitmq/plugins/']

Before, I was using the lifecycle to throw a wget to the download url and then unzip and copy the file, but I think the above is more "elegant"

lifecycle:
          postStart:
                  exec:
                    command: ['sh', '-c', 'wget https://dl.bintray.com/rabbitmq/community-plugins/3.7.x/rabbitmq_delayed_message_exchange/rabbitmq_delayed_message_exchange-20171201-3.7.x.zip && unzip rabbitmq_delayed_message_exchange-20171201-3.7.x.zip -d /opt/rabbitmq/plugins/']

Answer 3

I have used following life cycle hook for enabling plugin:

lifecycle:
  postStart:
    exec:
      command: ["/bin/sh", "-c", "rabbitmq-plugins --offline enable rabbitmq_management rabbitmq_peer_discovery_k8s rabbitmq_prometheus"]

Need to use --offline flag to wait for rabbitmq to start and then enable.

Mounting didn't work for me. I was getting error in rabbitmq that not able to write file in the /etc/rabbitmq. Couldn't resolve it because of this https://github.com/kubernetes/kubernetes/pull/58720

Here you can see one sample manifest:

---
apiVersion: v1
kind: Service
metadata:
  name: rabbitmq
  namespace: develop
spec:
  type: ClusterIP
  selector:
    app: rabbitmq
    tier: core
  ports:
    - name: port-5672-tcp
      port: 5672
    - name: port-15672-tcp
      port: 15672

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rabbitmq
  namespace: develop
spec:
  replicas: 2
  selector:
    matchLabels:
      app: rabbitmq
      tier: core
  template:
    metadata:
      labels:
        app: rabbitmq
        tier: core
    spec:
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      volumes:
        - name: rabbitmq-storage
          persistentVolumeClaim:
            claimName: rabbitmq-pvc
      containers:
        - name: rabbitmq
          image: rabbitmq:3.8-management
          lifecycle:
            postStart:
              exec:
                command: ["/bin/sh", "-c", "rabbitmq-plugins --offline enable rabbitmq_management rabbitmq_peer_discovery_k8s rabbitmq_prometheus"]
          resources:
            requests:
              memory: 2Gi
              cpu: 1
            limits:
              memory: 2Gi
              cpu: 1
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 5672
            - containerPort: 15672
          volumeMounts:
            - name: rabbitmq-storage
              mountPath: "/var/lib/rabbitmq/"
          env:
            - name: RABBITMQ_DEFAULT_USER
              valueFrom:
                secretKeyRef:
                  name: rabbitmq-username
                  key: RABBITMQ__USERNAME
            - name: RABBITMQ_DEFAULT_PASS
              valueFrom:
                secretKeyRef:
                  name: rabbitmq-password
                  key: RABBITMQ__PASSWORD
      nodeSelector:
        type: ultrafastest