How to pass extra configuration to RabbitMQ with Helm?

Question

I'm using this chart: https://github.com/helm/charts/tree/master/stable/rabbitmq to deploy a cluster of 3 RabbitMQ nodes on Kubernetes. My intention is to have all the queues mirrored within 2 nodes in the cluster.

Here's the command I use to run Helm: helm install --name rabbitmq-local -f rabbitmq-values.yaml stable/rabbitmq

And here's the content of rabbitmq-values.yaml:

persistence:
  enabled: true

resources:
  requests:
    memory: 256Mi
    cpu: 100m

replicas: 3

rabbitmq:
  extraConfiguration: |-
    {
      "policies": [
        {
          "name": "queue-mirroring-exactly-two",
          "pattern": "^ha\.",
          "vhost": "/",
          "definition": {
            "ha-mode": "exactly",
            "ha-params": 2
          }
        }
      ]
    }

However, the nodes fail to start due to some parsing errors, and they stay in crash loop. Here's the output of kubectl logs rabbitmq-local-0:

BOOT FAILED
===========

Config file generation failed:
=CRASH REPORT==== 23-Jul-2019::15:32:52.880991 ===
  crasher:
    initial call: lager_handler_watcher:init/1
    pid: <0.95.0>
    registered_name: []
    exception exit: noproc
      in function  gen:do_for_proc/2 (gen.erl, line 228)
      in call from gen_event:rpc/2 (gen_event.erl, line 239)
      in call from lager_handler_watcher:install_handler2/3 (src/lager_handler_watcher.erl, line 117)
      in call from lager_handler_watcher:init/1 (src/lager_handler_watcher.erl, line 51)
      in call from gen_server:init_it/2 (gen_server.erl, line 374)
      in call from gen_server:init_it/6 (gen_server.erl, line 342)
    ancestors: [lager_handler_watcher_sup,lager_sup,<0.87.0>]
    message_queue_len: 0
    messages: []
    links: [<0.90.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 610
    stack_size: 27
    reductions: 228
  neighbours:

15:32:53.679 [error] Syntax error in /opt/bitnami/rabbitmq/etc/rabbitmq/rabbitmq.conf after line 14 column 1, parsing incomplete
=SUPERVISOR REPORT==== 23-Jul-2019::15:32:53.681369 ===
    supervisor: {local,gr_counter_sup}
    errorContext: child_terminated
    reason: killed
    offender: [{pid,<0.97.0>},
               {id,gr_lager_default_tracer_counters},
               {mfargs,{gr_counter,start_link,
                                   [gr_lager_default_tracer_counters]}},
               {restart_type,transient},
               {shutdown,brutal_kill},
               {child_type,worker}]
=SUPERVISOR REPORT==== 23-Jul-2019::15:32:53.681514 ===
    supervisor: {local,gr_param_sup}
    errorContext: child_terminated
    reason: killed
    offender: [{pid,<0.96.0>},
               {id,gr_lager_default_tracer_params},
               {mfargs,{gr_param,start_link,[gr_lager_default_tracer_params]}},
               {restart_type,transient},
               {shutdown,brutal_kill},
               {child_type,worker}]

If I remove the rabbitmq.extraConfiguration part, the nodes start properly, so it must be something wrong with the way I'm typing in the policy. Any idea what I'm doing wrong?

Thank you.

score 4 · Accepted Answer · answered Jul 30 '19 at 12:41

According to https://github.com/helm/charts/tree/master/stable/rabbitmq#load-definitions, it is possible to link a JSON configuration as extraConfiguration. So we ended up with this setup that works:

rabbitmq-values.yaml:

rabbitmq:
  loadDefinition:
    enabled: true
    secretName: rabbitmq-load-definition
  extraConfiguration:
    management.load_definitions = /app/load_definition.json

rabbitmq-secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: rabbitmq-load-definition
type: Opaque
stringData:
  load_definition.json: |-
    {
      "vhosts": [
        {
          "name": "/"
        }
      ],
      "policies": [
        {
          "name": "queue-mirroring-exactly-two",
          "pattern": "^ha\.",
          "vhost": "/",
          "definition": {
            "ha-mode": "exactly",
            "ha-params": 2
          }
        }
      ]
    }

The secret must be loaded into Kubernetes before the Helm chart is played, which goes something like this: kubectl apply -f ./rabbitmq-secret.yaml.

the `loadDefinition.secretName` is now `loadDefinition.existingSecret` — MrFabio, Oct 11 '22 at 16:54

score 3 · Answer 2 · answered Aug 26 '20 at 04:40

You can use config default of HelmChart

If needed, you can use extraSecrets to let the chart create the secret for you. This way, you don't need to manually create it before deploying a release. For example :

extraSecrets:
  load-definition:
    load_definition.json: |
      {
        "vhosts": [
          {
            "name": "/"
          }
        ]
      }
rabbitmq:
  loadDefinition:
    enabled: true
    secretName: load-definition
  extraConfiguration: |
    management.load_definitions = /app/load_definition.json

https://github.com/helm/charts/tree/master/stable/rabbitmq

score 0 · Answer 3 · answered Jul 24 '19 at 05:38

0

Instead of using extraConfiguration, use advancedConfiguration, you should put all these info in this section as it is for classic config format (erlang)

answered Jul 24 '19 at 05:38

Pawan Kumar

830
1
9
15

1

This is the right answer. However, I must note that the configuration added by the user looks like JSON instead of erlang config format. You can add configuration options in two different places: - `extraConfiguration`, using the sysctl format - `advancedConfiguration`, using the erlang config format. The format provided by the user is not valid. – Ale Jul 24 '19 at 16:05
Thank you both for your answers. I was trying however to load the config as JSON. The answer I posted below works in that aspect. – Bogdan Minciu Jul 30 '19 at 12:43
1

Would it be possible to extend your answer with a working example of how to present the OP's config using the advancedConfiiguration option you suggest? – starfry Aug 06 '20 at 09:30
@PawanKumar Please add some information in your answer, please, if you can – VMAtm Dec 28 '21 at 08:03

How to pass extra configuration to RabbitMQ with Helm?

3 Answers3