0

I was trying to debug some mount problems and the mount logs led me to paths under /var/lib/kubelet/pods, i.e

/var/lib/kubelet/pods/f6affad1-941d-4df1-a0b7-38e3f2ab99d5/volumes/kubernetes.io~nfs/my-pv-e0dbe341a6fe475c9029fb372e

How can I map the guid of the root directory under pods to the actual running pod or container?

(f6affad1-941d-4df1-a0b7-38e3f2ab99d5 in the example above)

I don't see any correlation to the values returned by kubectl or crictl.

Mugen
  • 8,301
  • 10
  • 62
  • 140

2 Answers2

5

They're the .metadata.uid of the Pod; one can map them back by using your favorite mechanism for querying all pods and filtering on its .metadata.uid, and optionally restricting to just those pods scheduled on that Node if you have a so many Pods as to make the -A infeasible

for d in /var/lib/kubelet/pods/*; do
  p_u=$(basename "$d")
  kubectl get po -A -o json | \
    jq --arg pod_uuid "$p_u" -r '.items[] 
      | select(.metadata.uid == $pod_uuid) 
      | "uuid \($pod_uuid) is \(.metadata.name)"'
done

I'm sure there is a -o jsonpath= or -o gotemplate= form that removes the need for jq but that'd be a lot more work to type out in a textarea

with regard to your crictl question, I don't this second have access to my containerd cluster, but the docker based one labels the local containers with io.kubernetes.pod.uid so I would guess containerd does something similar:

            "Labels": {
                "annotation.io.kubernetes.container.hash": "e44bee94",
                "annotation.io.kubernetes.container.restartCount": "4",
                "annotation.io.kubernetes.container.terminationMessagePath": "/dev/termination-log",
                "annotation.io.kubernetes.container.terminationMessagePolicy": "File",
                "annotation.io.kubernetes.pod.terminationGracePeriod": "30",
                "io.kubernetes.container.logpath": "/var/log/pods/kube-system_storage-provisioner_b4aa3b1c-62c1-4661-a302-4c06b305b7c0/storage-provisioner/4.log",
                "io.kubernetes.container.name": "storage-provisioner",
                "io.kubernetes.docker.type": "container",
                "io.kubernetes.pod.name": "storage-provisioner",
                "io.kubernetes.pod.namespace": "kube-system",
                "io.kubernetes.pod.uid": "b4aa3b1c-62c1-4661-a302-4c06b305b7c0",
                "io.kubernetes.sandbox.id": "3950ec60121fd13116230cad388a4c6c4e417c660b7da475436f9ad5c9cf6738"
            }
mdaniel
  • 31,240
  • 5
  • 55
  • 58
1

A simpler way to map to the pod name that does not require kubectl and relies on a container name saved in filesystem is:

ls -lad /var/lib/kubelet/pods/*/containers/*

# Sample output:
# /var/lib/kubelet/pods/0a82722a-57e7-4a32-b0e7-1d3864b98c98/containers/worker
# /var/lib/kubelet/pods/148c7195-c3ab-4750-afb4-e6fb34948ed5/containers/dashboard
# /var/lib/kubelet/pods/6bb42e72-902d-49b1-8beb-0e885e2708fc/containers/kube-proxy
# ...

find /var/lib/kubelet/pods/ -maxdepth 3 -name "*dnsutils*"

# Sample output:
# /var/lib/kubelet/pods/a18df6c7-3a37-4bc0-8593-026145f4c4b6/containers/dnsutils
Aleksejs Spiridonovs
  • 424
  • 2
  • 6