How can we see cached images in kubernetes?

Question

I am using kops as kubernetes deployment.

I noticed that whenever an image with same tag number is entered in deployment file the system takes the previous image if imagepullpolicy is not set to always

Is there any way in which I can see all the cached images of a container in kubernetes environment ?

Like suppose I have an image test:56 currently running in a deployment and test:1 to test:55 were used previously, so does kubernetes cache those images ? and if yes where can those be found ?

Answer 1

• Comments on your environment:

  I noticed that whenever an image with same tag number is entered in deployment file the system takes the previous image if imagepullpolicy is not set to always

A pre-pulled image can be used to preload certain images for speed or as an alternative to authenticating to a private registry, optimizing performance.

The docker will always cache all images that were used locally.

Since you are using EKS, keep in mind that if you have node health management (meaning a node will be replaced if it fails) the new node won't have the images cached from the old one so it's always a good idea to store your images on a Registry like your Cloud Provider Registry or a local registry.

• Let's address your first question:

  Is there any way in which I can see all the cached images of a container in kubernetes environment ?

Yes, you must use docker images to list the images stored in your environment.

• Second question:

  Like suppose I have an image test:56 currently running in a deployment and test:1 to test:55 were used previously, so does Kubernetes cache those images ? and if yes where can those be found ?

I prepared an example for you:

• I deployed several pods based on the official busybox image:

$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.28.4
pod/busy284 created
$ kubectl run busy293 --generator=run-pod/v1 --image=busybox:1.29.3
pod/busy284 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.28
pod/busy28 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.29
pod/busy29 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.30
pod/busy284 created
$ kubectl run busybox --generator=run-pod/v1 --image=busybox
pod/busybox created

Now let's check the images stored in docker images

$ docker images
REPOSITORY                                TAG                   IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy                     v1.17.3               ae853e93800d        5 weeks ago         116MB
k8s.gcr.io/kube-controller-manager        v1.17.3               b0f1517c1f4b        5 weeks ago         161MB
k8s.gcr.io/kube-apiserver                 v1.17.3               90d27391b780        5 weeks ago         171MB
k8s.gcr.io/kube-scheduler                 v1.17.3               d109c0821a2b        5 weeks ago         94.4MB
kubernetesui/dashboard                    v2.0.0-beta8          eb51a3597525        3 months ago        90.8MB
k8s.gcr.io/coredns                        1.6.5                 70f311871ae1        4 months ago        41.6MB
k8s.gcr.io/etcd                           3.4.3-0               303ce5db0e90        4 months ago        288MB
kubernetesui/metrics-scraper              v1.0.2                3b08661dc379        4 months ago        40.1MB
busybox                                   latest                83aa35aa1c79        10 days ago         1.22MB
busybox                                   1.30                  64f5d945efcc        10 months ago       1.2MB
busybox                                   1.29                  758ec7f3a1ee        15 months ago       1.15MB
busybox                                   1.29.3                758ec7f3a1ee        15 months ago       1.15MB
busybox                                   1.28                  8c811b4aec35        22 months ago       1.15MB
busybox                                   1.28.4                8c811b4aec35        22 months ago       1.15MB

You can see all the pushed images listed.

It's good to clean old resources from your system using the command docker system prune to free space on your server from time to time.

If you have any doubt, let me know in the comments.

Answer 2

Is there any way in which I can see all the cached images of a container in kubernetes environment ?

This is the basic question I had, but when docker was not in use as the container runtime. And, in various environments docker will not be available since other container runtimes (CRI) will be in use. On an ubuntu system using the CRI-O container runtime, you can use the cri-tools package, which provide crictl. To use this you must be root, on any of the nodes in your cluster.

$ sudo crictl images
IMAGE                           TAG                 IMAGE ID            SIZE
docker.io/library/busybox       latest              66ba00ad3de86       5.1MB
docker.io/library/nginx         1.16.1              dfcfd8e9a5d38       131MB
docker.io/weaveworks/weave-kube           latest              62fea85d60522       89.8MB
docker.io/weaveworks/weave-npc            latest              690c3345cc9c3       39.7MB
registry.k8s.io/etcd                      3.5.6-0             fce326961ae2d       301MB
registry.k8s.io/kube-apiserver            v1.26.2             63d3239c3c159       135MB
<snip>

cri-tools can be installed like sudo apt-get install cri-tools. crictl also provides typical commands from a CRI like exec, attach, ps but many of these are better done using the equivalent kubectl command.