Backup/Recover Kubernetes' current state (Namespaces/Pods/Controllers/etc)

Question

I would like to be able to get a description of my current state of the cluster so that in the future I would be able to recover from a failure. Aside from recreating all of the services from source/cli individually, what solutions are available?

Answer 1

Update: this is a really old method. We now have much better tools to backup k8s clusters, like velero

I'm using a bash script from CoreOS team, with small adjustments, that works pretty good. I'm using it more for cluster migration, but at some level can be used for backups too.

for ns in $(kubectl get ns --no-headers | cut -d " " -f1); do
  if { [ "$ns" != "kube-system" ]; }; then
  kubectl --namespace="${ns}" get --export -o=json svc,rc,rs,deployments,cm,secrets,ds,petsets | \
jq '.items[] |
    select(.type!="kubernetes.io/service-account-token") |
    del(
        .spec.clusterIP,
        .metadata.uid,
        .metadata.selfLink,
        .metadata.resourceVersion,
        .metadata.creationTimestamp,
        .metadata.generation,
        .status,
        .spec.template.spec.securityContext,
        .spec.template.spec.dnsPolicy,
        .spec.template.spec.terminationGracePeriodSeconds,
        .spec.template.spec.restartPolicy
    )' >> "./my-cluster.json"
  fi
done

In case you need to revocer the state after, you just need to execute kubectl create -f ./my-cluster.json

Answer 2

I'd recommend Heptio Ark - https://github.com/heptio/ark. It's a general purpose disaster recovery solution for Kubernetes. It will back up all of your resources inside your cluster (pods, deployments, etc), and it can also take snapshots of your persistent volumes.

(disclaimer: I work for Heptio on Ark)