Mounting a configuration file structure in job pod

Question

In our application we will have a Kubernetes job that will bootstrap microservices based on a set of (legacy) configuration files. These are in the legacy application packaged as a file structure, and we'd like to keep this so that we don't need to maintain different packaging of the configuration files for the Kubernetes variant and the legacy variant, and there is a lot of legacy code that we still want to use that expects this file structure.

My approach was that I wanted to mount a ConfigMap in the job pod since I naively assumed you could map a entire folder tree, but a ConfigMap is a flat key-value storage.

We're using Helm, so the idea is that the microservice will include the job and apply it with hooks. What would be a good way to map the configuration file structure into the job pod? Can you easily create a volume, populate it with the file strucutre, and then mount it in the job pod?

Answer 1

You can create a configmap from a file or directory and add configmap data to a volume. This will create read-only files from configmap keys in mounted directory. However, it's not possible to keep directory structure in Configmap, as it will fail with error

... a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name',  or 'KEY_NAME',  or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+')

configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
data:
  some.properties: |
    foo=bar
  other.properties: |
    baz=qux

pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
    - name: test-container
      image: k8s.gcr.io/busybox
      args:
      - /bin/sh
      - -c
      - ls -la /etc/config; cat /etc/config/some/some.properties; cat /etc/config/other.properties
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: special-config
        items:
        - key: some.properties
          path: some/some.properties
        - key: other.properties
          path: other.properties

checking logs with kubectl logs test

 drwxrwxrwx    3 root     root            95 May 29 09:14 .
 drwxr-xr-x    1 root     root            20 May 29 09:14 ..
 drwxr-xr-x    3 root     root            42 May 29 09:14 ..2020_05_29_09_14_34.418889646
 lrwxrwxrwx    1 root     root            31 May 29 09:14 ..data -> ..2020_05_29_09_14_34.418889646
 lrwxrwxrwx    1 root     root            23 May 29 09:14 other.properties -> ..data/other.properties
 lrwxrwxrwx    1 root     root            11 May 29 09:14 some -> ..data/some
 foo=bar
 baz=qux

Answer 2

There are many ways to achieve the result, some are better than the other so I am just going mention the better ones.

You could in theory use configmaps to do it but you also mentioned "... there is a lot of legacy code" and because of that you need to aware that configmaps have 1MB limit (to be more specific I should mention that this limit is enforced by etcd and not configmap itself). Now assuming this is not a problem you would need to hold each file as one key in configmap and then mount every file individually to its directory. Of cource you could automate this process byt its not ideal solution when having many files.

---

Now answering to your question:

Can you easily create a volume, populate it with the file strucutre, and then mount it in the job pod?

Probably the easiest way to achieve mentioned result would be by mounting emptyDir volume and populating it using initContainers. Init container could pull configuration folder (either from git repo or from some other file repository) and save it in emptyDit volume that would be later mounted to the actual container. Here is an example of how it would look like.

What you are suggesting is also valid solution but it requires the use of persistent volumes and it may not always be necessary. But if you decide to use it, all you need to do is use the same volume name and use retain for persistentVolumeReclaimPolicy field.

You also need to remember that init(hook) container code should be idempotent. In particular, code that writes to EmptyDirs(persistent volume) should be prepared for the possibility that an output files already exist.