Tanka/Jsonnet - Howto loop over templates/imports?

Question

i am trying to create multiple grafana-instances with slightly different config-files with tanka. The following works, as long as the the configmap.grafana_ini is in-place. But this becomes very unreadable with a growing config. So i am looking for a way to move the configmaps to their own file and import.

But if i move that to it's own file and use an import/str i am getting a "computed imports are not allowed" error or the instance-variable becomes unknown.

local tanka = import 'github.com/grafana/jsonnet-libs/tanka-util/main.libsonnet';
local helm = tanka.helm.new(std.thisFile);

local k = import 'github.com/grafana/jsonnet-libs/ksonnet-util/kausal.libsonnet';


(import 'config.libsonnet') +

{
  local configMap = k.core.v1.configMap,
  local container = k.core.v1.container,
  local stateful = k.apps.v1.statefulSet,
  local ingrdatasourcesess = k.networking.v1.ingress,
  local port = k.core.v1.containerPort,
  local service = k.core.v1.service,
  local pvc = k.core.v1.persistentVolumeClaim,

  local ports = [port.new('http', 3000)],

  grafana: {
    g(instance):: {

      local this = self,

      deployment:
        stateful.new(
          name='grafana-' + instance.handle,
          replicas=1,
          containers=[
            container.new(
              name='grafana-' + instance.handle,
              image=$._config.grafana.image + instance.theme + ':' + $._config.grafana.version
            )
            + container.withPorts(ports),
          ],
        )
        + stateful.metadata.withLabels({ 'io.kompose.service': 'grafana-' + instance.handle })
        + stateful.configMapVolumeMount(this.configMaps.grafana_ini, '/etc/grafana/grafana.ini', k.core.v1.volumeMount.withSubPath('grafana.ini'))
        + stateful.spec.withServiceName('grafana-' + instance.handle)
        + stateful.spec.selector.withMatchLabels({ 'io.kompose.service': 'grafana-' + instance.handle })
        + stateful.spec.template.metadata.withLabels({ 'io.kompose.service': 'grafana-' + instance.handle })
        + stateful.spec.template.spec.withImagePullSecrets({
          name: 'registry.gitlab.com',
        })
        + stateful.spec.template.spec.withRestartPolicy('Always'),    

      service:
        k.util.serviceFor(self.deployment)
        + service.mixin.spec.withType('ClusterIP'),

      configMaps: {
        grafana_ini:
          configMap.new(
            'grafana-ini-' + instance.handle, {
              'grafana.ini': std.manifestIni(
                {
                  main: {
                    app_mode: 'production',
                    instance_name: instance.handle,
                  },
                  sections: {
                    server: {
                      protocol: 'http',
                      http_port: '3000',
                      domain: 'dashboard.' + $._config.ingress.realm + '.' + $._config.ingress.tld + '/' + instance.handle + '/',
                      root_url: $._config.ingress.protocol + 'dashboard.' + $._config.ingress.realm + '.' + $._config.ingress.tld + '/' + instance.handle + '/',
                      serve_from_sub_path: true,
                    },
                
                  },
                }
              ),
            }
          ),
      },
    },
    deploys: [self.g(instance) for instance in $._config.grafana.instances],
  },
}

Here is the config-part:

{
  _config+:: {
    grafana+: {
      image: 'registry.gitlab.com/xxx/frontend/grafana/',
      version: 'v7.3.7',
      client_secret: 'xyz',
      adminusername: 'admin',
      adminpassword: 'admin',
      instances: [
        {
          name: "xxx's Grafana",
          handle: 'xyz',
          theme: 'xxx',
          alerting: 'false',
          volume_size: '200M',
          default: true,
          allow_embedding: false,
          public: 'false',
          secret_key: 'xxxx',
          email: {
            host: '',
            user: '',
            password: '',
            from_address: '',
            from_name: '',
          },
          datasources: [
            {
              name: 'xxx Showcase',
              type: 'influxdb',
              access: 'proxy',
              url: 'http://influx:8086',
              database: 'test123',
              user: 'admin',
              password: 'admin',
              editable: false,
              isDefault: false,
              version: 1              
            },
          ],
          dashboards: [
            {
              src: 'provisioning/dashboards/xxx_showcase_dashboard.json',
              datasource: 'xxx Showcase',
              title: 'xxx office building',
              template: true,
            },
          ],
        },
      ],
    },
  },
}

EDITED, as suggested by 2nd post.

greetings, strowi

There are a couple issues w/your question: 1) Please also add an excerpt from `config-ini.libsonnet` else it's hard to get what are doing there; 2) In the main code snipped you're calling `my_deploy()` which is not implemented there, FWIW looks like bits from https://stackoverflow.com/questions/73108252/ — jjo, Aug 22 '22 at 18:06
Thanks for taking a look @jjo, the original was quite big (about 200 lines) and i guess i failed at trying to minimize while keeping the detail. I updated the post accordingly with your suggestions. — strowi, Aug 23 '22 at 06:59

score 0 · Accepted Answer · answered Aug 23 '22 at 12:51

Thanks for clarifying, find below a possible solution, note I trimmed down parts of your original files, for better readability.

I think that the main highlight here is the iniFile() function, so that we can explicitly pass (config, instance) to it.

main.jsonnet

local tanka = import 'github.com/grafana/jsonnet-libs/tanka-util/main.libsonnet';
local helm = tanka.helm.new(std.thisFile);

local k = import 'github.com/grafana/jsonnet-libs/ksonnet-util/kausal.libsonnet';


(import 'config.libsonnet') +

{
  local configMap = k.core.v1.configMap,
  local container = k.core.v1.container,
  local stateful = k.apps.v1.statefulSet,
  local ingrdatasourcesess = k.networking.v1.ingress,
  local port = k.core.v1.containerPort,
  local service = k.core.v1.service,
  local pvc = k.core.v1.persistentVolumeClaim,

  local ports = [port.new('http', 3000)],

  grafana: {
    g(instance):: {

      local this = self,

      /* <snip...> */
      configMaps: {
        local inilib = import 'ini.libsonnet',
        grafana_ini:
          configMap.new(
            'grafana-ini-' + instance.handle, inilib.iniFile($._config, instance)
          ),
      },
    },
    deploys: [self.g(instance) for instance in $._config.grafana.instances],
  },
}

config.libsonnet

{
  _config+:: {
    // NB: added below dummy ingress field
    ingress:: {
      realm:: 'bar',
      tld:: 'foo.tld',
      protocol:: 'tcp',
    },
    grafana+: {
      image: 'registry.gitlab.com/xxx/frontend/grafana/',
      version: 'v7.3.7',
      client_secret: 'xyz',
      adminusername: 'admin',
      adminpassword: 'admin',
      instances: [
        {
          name: "xxx's Grafana",
          handle: 'xyz',
          /* <snip...> */
        },
      ],
    },
  },
}

ini.libsonnet

{
  iniFile(config, instance):: {
    'grafana.ini': std.manifestIni(
      {
        main: {
          app_mode: 'production',
          instance_name: instance.handle,
        },
        sections: {
          server: {
            protocol: 'http',
            http_port: '3000',
            domain: 'dashboard.' + config.ingress.realm + '.' + config.ingress.tld + '/' + instance.handle + '/',
            root_url: config.ingress.protocol + 'dashboard.' + config.ingress.realm + '.' + config.ingress.tld + '/' + instance.handle + '/',
            serve_from_sub_path: true,
          },

        },
      }
    ),
  },
}

Thank you, that definitely works;) So i need to wrap these in functions - as opposed to some kind of inheritance. (It's a bit hard to find good documentation). Now i am thinking about abstracting this kind of function to make it reusable - not sure this is possible, i would need sth. like ini.inifile(config, instance, file, replacements) i guess.. — strowi, Aug 24 '22 at 07:10
As always, there are several ways to implement the same resulting manifest. I _think_ that approaching with functions makes it clearer and more readable than say adding fields to the object to allow referring them with e.g. `self.instance` in `ini.libsonnet` (instead of a function argument, as implemented in my answer). — jjo, Aug 24 '22 at 15:26

Tanka/Jsonnet - Howto loop over templates/imports?

1 Answers1

main.jsonnet

config.libsonnet

ini.libsonnet