2

I'm attempting to migrate a helm/kubernetes codebase to dhall-kubernetes. Dhall is typed, so I need to provide full records with optional fields set to null if there were not set. So I'm looking for something like kubectl get objname id -o yaml, but I need it to output all optional fields like fieldName: null. Is there a way to accomplish this? I don't know how to do it, so as a plan B I wrote dhall-default and I attempt to approach it in a different way.

dredozubov
  • 715
  • 1
  • 6
  • 11
  • 1
    Sorry, I don't know much about kubernetes, and I'm not sure I really understand your question. Will the optional fields only serve to combine them with the required ones? In that case it might be easier to use the schemas provided by `dhall-kubernetes`. That way you only have to provide the required fields. For example: `let ServiceReference = https://raw.githubusercontent.com/dhall-lang/dhall-kubernetes/master/1.17/schemas/io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.ServiceReference.dhall in ServiceReference::{ name = "x", namespace = "y" }` – sjakobi Apr 03 '20 at 11:56
  • 1
    Oh, and you can of course access a schema's `default`! `echo 'let ServiceReference = https://raw.githubusercontent.com/dhall-lang/dhall-kubernetes/master/1.17/schemas/io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.ServiceReference.dhall in ServiceReference.default' | dhall-to-json --preserve-null --compact` produces `{"path":null,"port":null}` – sjakobi Apr 03 '20 at 11:59
  • 1
    In general you can find these defaults in https://github.com/dhall-lang/dhall-kubernetes/tree/master/1.17/defaults or https://github.com/dhall-lang/dhall-kubernetes/blob/master/defaults.dhall – sjakobi Apr 03 '20 at 12:03
  • Thanks for mentioning this. It's helpful, but it's not what I'm looking for here. What I'm trying to undestand is how to automate the migration process somehow to extract the data from kubernetes in a way it'll be compatible with dhall. – dredozubov Apr 03 '20 at 12:16
  • So you want to take your existing kubernetes configs in YAML and convert them to Dhall. Is that right? Otherwise please clarify your question. – sjakobi Apr 03 '20 at 12:25
  • Yes. I have a codebase with a mix of helm charts and kubernetes YAML configs. I'm trying to come out with a way of converting them to dhall before refactoring it. – dredozubov Apr 03 '20 at 12:31
  • 2
    So are you using `yaml-to-dhall`? It doesn't require that optional fields are set. For example `echo '{"name":"x","namespace":"y"}' | json-to-dhall 'let ServiceReference = https://raw.githubusercontent.com/dhall-lang/dhall-kubernetes/master/1.17/schemas/io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.ServiceReference.dhall in ServiceReference.Type'` produces `{ name = "x", namespace = "y", path = None Text, port = None Natural }` – sjakobi Apr 03 '20 at 12:34
  • Oh, cool trick! I think it can be counted as an answer here, although it's a different approach to what I imagined. – dredozubov Apr 03 '20 at 12:55

1 Answers1

4

I'll turn @sjakobi's solution into an answer like @dredozubov suggested

You can pass the desired type from dhall-kubernetes to yaml-to-dhall and it will generate the equivalent Dhall code, albeit without any simplifications.

As an example, suppose that you have the following Kubernetes resource:

# ./nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      name: nginx
  template:
    metadata:
      name: nginx
    spec:
      containers:
        - image: nginx:1.15.3
          name: nginx
          ports:
            - containerPort: 80

... and the following Dhall type for a Kubernetes deployment:

-- ./Deployment.dhall

let kubernetes = https://raw.githubusercontent.com/dhall-lang/dhall-kubernetes/506d633e382872346927b8cb9884d8b7382e6cab/package.dhall

in  kubernetes.Deployment.Type

Then you can translate the YAML to Dhall by running:

$ yaml-to-dhall --file ./nginx.yaml ./Deployment.dhall

The output is a bit large (~1300 lines), because yaml-to-dhall doesn't yet take advantage of support for default values, so I won't include the output here.

If you pipe the result back into dhall-to-yaml then you will get the original Resource (albeit with fields sorted):

$ yaml-to-dhall --file ./nginx.yaml ./Deployment.dhall | dhall-to-yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      name: nginx
  template:
    metadata:
      name: nginx
    spec:
      containers:
        - image: nginx:1.15.3
          name: nginx
          ports:
            - containerPort: 80

... and if you supply the --preserve-null option to dhall-to-yaml it will preserve all null fields as the question requests:

$ yaml-to-dhall --file ./nginx.yaml ./Deployment.dhall | dhall-to-yaml --preserve-null 
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations: null
  clusterName: null
  creationTimestamp: null
  deletionGracePeriodSeconds: null
  deletionTimestamp: null
  finalizers: null
  generateName: null
  generation: null
  labels: null
  managedFields: null
  name: nginx
  namespace: null
  ownerReferences: null
  resourceVersion: null
  selfLink: null
  uid: null
spec:
  minReadySeconds: null
  paused: null
  progressDeadlineSeconds: null
  replicas: 2
  revisionHistoryLimit: null
  selector:
    matchExpressions: null
    matchLabels:
      name: nginx
  strategy: null
  template:
    metadata:
      annotations: null
      clusterName: null
      creationTimestamp: null
      deletionGracePeriodSeconds: null
      deletionTimestamp: null
      finalizers: null
      generateName: null
      generation: null
      labels: null
      managedFields: null
      name: nginx
      namespace: null
      ownerReferences: null
      resourceVersion: null
      selfLink: null
      uid: null
    spec:
      activeDeadlineSeconds: null
      affinity: null
      automountServiceAccountToken: null
      containers:
        - args: null
          command: null
          env: null
          envFrom: null
          image: nginx:1.15.3
          imagePullPolicy: null
          lifecycle: null
          livenessProbe: null
          name: nginx
          ports:
            - containerPort: 80
              hostIP: null
              hostPort: null
              name: null
              protocol: null
          readinessProbe: null
          resources: null
          securityContext: null
          startupProbe: null
          stdin: null
          stdinOnce: null
          terminationMessagePath: null
          terminationMessagePolicy: null
          tty: null
          volumeDevices: null
          volumeMounts: null
          workingDir: null
      dnsConfig: null
      dnsPolicy: null
      enableServiceLinks: null
      ephemeralContainers: null
      hostAliases: null
      hostIPC: null
      hostNetwork: null
      hostPID: null
      hostname: null
      imagePullSecrets: null
      initContainers: null
      nodeName: null
      nodeSelector: null
      overhead: null
      preemptionPolicy: null
      priority: null
      priorityClassName: null
      readinessGates: null
      restartPolicy: null
      runtimeClassName: null
      schedulerName: null
      securityContext: null
      serviceAccount: null
      serviceAccountName: null
      shareProcessNamespace: null
      subdomain: null
      terminationGracePeriodSeconds: null
      tolerations: null
      topologySpreadConstraints: null
      volumes: null
status: null
Gabriella Gonzalez
  • 34,863
  • 3
  • 77
  • 135
  • If you are posting asnswer based on someone else comment, you should mark it as `Community Wiki` as per: https://stackoverflow.com/help/privileges/community-wiki – PjoterS Apr 03 '20 at 15:51
  • I suggest we wait for a bit before I mark this as an accepted answer because it doesn't give a direct answer to the stated question, but introduces a different way of approaching this. – dredozubov Apr 06 '20 at 09:42
  • @dredozubov: I updated the answer to explain how to preserve `null`s when converting back to YAML – Gabriella Gonzalez Apr 08 '20 at 05:08