0

I have a backup job that runs on a cron schedule. Each time the job is run a Kubernetes pod is created, puts backup data into a bound PVC, and completes. I need to be able to manually copy data out of the PVC and put it somewhere else after the job has finished (when the pod is in the completed state).

At this point, the PVC is still listed as Bound, but the pod is listed as Completed. So trying to run kubectl cp my-pod:/ data/ returns an error that kubectl is unable to exec inside a completed pod, and trying to log into the pod with kubectl exec -it my-pod -- bash similarly returns the same error.

Is there a way to access the PVC data directly after the pod completes?

A. Duff
  • 4,097
  • 7
  • 38
  • 69
  • 1
    You can launch another Pod that mount the same PVC – Jonas May 17 '21 at 18:34
  • @Jonas that's highly dependent on your backend storage. Even storage system such as AWS EBS that technically support multi-attach do not in k8s. – jordanm May 17 '21 at 18:54
  • 1
    I would recommend looking into CSI if you storage system supports snapshotting, otherwise have a look at off the shelf backup products like valero or stash. – jordanm May 17 '21 at 18:57
  • @jordanm even with EBS this is easy to mount in multiple Pods – Jonas May 17 '21 at 19:17
  • @Jonas only if the pods happen to be on the same instance. k8s does not support EBS multi attach: https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues/449 – jordanm May 17 '21 at 19:19
  • @jordanm yes, they need to be on the same instance if both is running, but if one is completed, it does not need to be on the same node, but probably in the same AZ. You should not need multi-attach here, this also works with ReadWriteOnce. – Jonas May 17 '21 at 19:21
  • After this job is done processing, cant you add another command to your pod script to rsync/scp/sftp your data out to a remote directory? – Rakesh Gupta May 17 '21 at 21:20

0 Answers0