kubernetes cannot pull a public image

Question

kubernetes cannot pull a public image. Standard images like nginx are downloading successfully, but my pet project is not downloading. I'm using minikube for launch kubernetes-cluster

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-gateway-deploumnet
  labels:
    app: api-gateway
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-gateway
  template:
    metadata:
      labels:
        app: api-gateway
    spec:
      containers:
      - name: api-gateway
        image: creatorsprodhouse/api-gateway:latest
        imagePullPolicy: Always
        ports:
        - containerPort: 80

when I try to create a deployment I get an error that kubernetes cannot download my public image.

$ kubectl get pods

result:

NAME                                      READY   STATUS             RESTARTS   AGE
api-gateway-deploumnet-599c784984-j9mf2   0/1     ImagePullBackOff   0          13m
api-gateway-deploumnet-599c784984-qzklt   0/1     ImagePullBackOff   0          13m
api-gateway-deploumnet-599c784984-csxln   0/1     ImagePullBackOff   0          13m

$ kubectl logs api-gateway-deploumnet-599c784984-csxln

result

Error from server (BadRequest): container "api-gateway" in pod "api-gateway-deploumnet-86f6cc5b65-xdx85" is waiting to start: trying and failing to pull image

What could be the problem? The standard images are downloading but my public one is not. Any help would be appreciated.

EDIT 1

$ api-gateway-deploumnet-599c784984-csxln

result:

Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  8m22s                  default-scheduler  Successfully assigned default/api-gateway-deploumnet-849899786d-mq4td to minikube
  Warning  Failed     3m8s                   kubelet            Failed to pull image "creatorsprodhouse/api-gateway:latest": rpc error: code = Unknown desc = context deadline exceeded
  Warning  Failed     3m8s                   kubelet            Error: ErrImagePull
  Normal   BackOff    3m7s                   kubelet            Back-off pulling image "creatorsprodhouse/api-gateway:latest"
  Warning  Failed     3m7s                   kubelet            Error: ImagePullBackOff
  Normal   Pulling    2m53s (x2 over 8m21s)  kubelet            Pulling image "creatorsprodhouse/api-gateway:latest"

EDIT 2

If I try to download a separate docker image, it's fine

$ docker pull creatorsprodhouse/api-gateway:latest

result:

Digest: sha256:e664a9dd9025f80a3dd60d157ce1464d4df7d0f8a00538e6a137d44f9f9f12aa
Status: Downloaded newer image for creatorsprodhouse/api-gateway:latest
docker.io/creatorsprodhouse/api-gateway:latest

EDIT 3 After advice to restart minikube

$ minikube stop

$ minikube delete --purge

$ minikube start --cni=calico

I started the pods.


Events:
  Type     Reason                  Age    From               Message
  ----     ------                  ----   ----               -------
  Normal   Scheduled               4m28s  default-scheduler  Successfully assigned default/api-gateway-deploumnet-849899786d-bkr28 to minikube
  Warning  FailedCreatePodSandBox  4m27s  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "7e112c92e24199f268ec9c6f3a6db69c2572c0751db9fd57a852d1b9b412e0a1" network for pod "api-gateway-deploumnet-849899786d-bkr28": networkPlugin cni failed to set up pod "api-gateway-deploumnet-849899786d-bkr28_default" network: failed to set bridge addr: could not add IP address to "cni0": permission denied, failed to clean up sandbox container "7e112c92e24199f268ec9c6f3a6db69c2572c0751db9fd57a852d1b9b412e0a1" network for pod "api-gateway-deploumnet-849899786d-bkr28": networkPlugin cni failed to teardown pod "api-gateway-deploumnet-849899786d-bkr28_default" network: running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.85.0.34 -j CNI-57e7da7379b524635074e6d0 -m comment --comment name: "crio" id: "7e112c92e24199f268ec9c6f3a6db69c2572c0751db9fd57a852d1b9b412e0a1" --wait]: exit status 2: iptables v1.8.4 (legacy): Couldn't load target `CNI-57e7da7379b524635074e6d0':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

Try descriing one of the pods, it might provide more details in the `Events` section: `kubectl describe pod api-gateway-deploumnet-599c784984-csxln` — Blender Fox, Jul 29 '22 at 08:26
Also one thing to check -- is your node an arm-based machine? — Blender Fox, Jul 29 '22 at 08:27
@BlenderFox Thank you. How can I check if the node is an arm-based machine? — MiyRon, Jul 29 '22 at 08:32
Thanks, that's not an arm machine then, so that's one possible cause eliminated. — Blender Fox, Jul 29 '22 at 08:43
It looks like your minikube cluster can't talk out: `Failed to pull image "creatorsprodhouse/api-gateway:latest": rpc error: code = Unknown desc = context deadline exceeded` — Blender Fox, Jul 29 '22 at 08:44
You were able to pull that image manually I see from your edit. So have you enabled a CNI like Weave or Calico in your minikube cluster? — Blender Fox, Jul 29 '22 at 08:45
@BlenderFox No. I'm a beginner and honestly this is the first time I've heard of CNI or Calico — MiyRon, Jul 29 '22 at 08:47
Certainly, how did you create the minikube cluster, from the command line or via a UI? — Blender Fox, Jul 29 '22 at 08:47
from the command line. i'm launched command "minikube start" — MiyRon, Jul 29 '22 at 08:48
Try deleting the minikube cluster (`minikube stop`, `minikube delete --purge`) then recreating it with a CNI (`minikube start --cni=calico`) then try deploying your deployment again. This should work as I use that same startup (plus a few extra switches) on my own local machine. — Blender Fox, Jul 29 '22 at 08:54
CNI stands for Container Network Interface https://www.cni.dev/ — Blender Fox, Jul 29 '22 at 09:10
@BlenderFox Unfortunately, it didn't help. Now in the describe pod events the following scary messages. I will add them to edit 3 — MiyRon, Jul 29 '22 at 09:14
Hmmm. Okay, then I suggest reverting back to the original setup you had and reporting an issue on the minikube project (https://github.com/kubernetes/minikube). Something is definitely not right with your setup, but I can't see anything obvious from your output — Blender Fox, Jul 29 '22 at 09:34
seems like you need to disable ipv6 `sysctl -w net.ipv6.conf.all.disable_ipv6=0` — Adiii, Jul 29 '22 at 09:34
@MiyRon : Is your issue resolved? If yes, can you provide the resolution steps you have followed and provide it as an answer for the greater visibility of the community. — Hemanth Kumar, Aug 05 '22 at 08:56

score 1 · Accepted Answer · answered Aug 09 '22 at 11:12

I could not solve the problem in the ways I was suggested. However, it worked when I ran minikube with a different driver

$ minikube start --driver=none

--driver=none means that the cluster will run on your host instead of the standard --driver=docker which runs the cluster in docker.

It is better to run minikube with --driver=docker as it is safer and easier, but it didn't work for me as I could not download my images. For me personally it is ok to use --driver=none although it is a bit dangerous.

In general, if anyone knows what the problem is, please answer my question. In the meantime you can try to run minikube cluster on your host with the command I mentioned above.

In any case, thank you very much for your attention!

kubernetes cannot pull a public image

1 Answers1

Linked