Azure Container Services: trying and failing to pull image

Question

I'm trying to deploy my k8s cluster. But when I do, it can't pull the image. Here's what I get when I run kubectl describe pods:

  Type     Reason      Age               From                   Message
  ----     ------      ----              ----                   -------
  Normal   BackOff     47m               kubelet, dc9ebacs9000  Back-off pulling image "tlk8s.azurecr.io/devicecloudwebapi:v1"
  Warning  FailedSync  9m (x3 over 47m)  kubelet, dc9ebacs9000  Error syncing pod
  Warning  Failed      9m                kubelet, dc9ebacs9000  Failed to pull image "tlk8s.azurecr.io/devicecloudwebapi:v1": [rpc error: code = 2 desc = failed to register layer: re-exec error: exit status 1: output: remove \\?\C:\ProgramData\docker\windowsfilter\930af9d006462c904d9114da95523cc441206db8bb568769f4f0612d3a96da5b\Files\Windows\System32\LogFiles\Scm\SCM.EVM: The system cannot find the file specified., rpc error: code = 2 desc = failed to register layer: re-exec error: exit status 1: output: remove \\?\C:\ProgramData\docker\windowsfilter\e30d44f97c53edf7e91c69f246fe753a84e4cb40899f472f75aae6e6d74b5c45\Files\Windows\System32\LogFiles\Scm\SCM.EVM: The system cannot find the file specified.]
  Normal   Pulling     9m (x3 over 2h)   kubelet, dc9ebacs9000  pulling image "tlk8s.azurecr.io/devicecloudwebapi:v1"

Here's what I get when I look at the individual pod:

Error from server (BadRequest): container "tl-api" in pod "tl-api-3363368743-d7kjq" is waiting to start: image can't be pulled

Here's my YAML file:

---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: tl-api
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: tl-api
    spec:
      containers:
      - name: tl-api
        image: tlk8s.azurecr.io/devicecloudwebapi:v1
        ports:
        - containerPort: 80
      imagePullSecrets:
      - name: acr-secret
      nodeSelector:
        beta.kubernetes.io/os: windows
---
apiVersion: v1
kind: Service
metadata:
  name: tl-api
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: tl-api

My docker images result:

REPOSITORY                                   TAG                            IMAGE ID            CREATED             SIZE
devicecloudwebapi                            latest                         ee3d9c3e231d        8 days ago          7.85GB
tlk8s.azurecr.io/devicecloudwebapi           v1                             ee3d9c3e231d        8 days ago          7.85GB
devicecloudwebapi                            dev                            bb33ab221910        8 days ago          7.76GB

What about the pod .yaml file? Can you show me? – Charles Xu Jul 12 '18 at 07:31 — Charles Xu, Jul 12 '18 at 07:31

André de Mattos Ferraz · Answer 1 · 2021-12-29T23:36:10.237

You must create a secret to your registry in kubectl:

kubectl create secret docker-registry <secret-name> \
--namespace <namespace> \
--docker-server=<container-registry-name>.azurecr.io \
--docker-username=<service-principal-ID> \
--docker-password=<service-principal-password>

More info: https://learn.microsoft.com/pt-br/azure/container-registry/container-registry-auth-kubernetes

Remember to set the "imagePullSecrets" into your spec.

apiVersion: v1
kind: Pod
metadata: #informaçoes internas do container
    name: mongodb-pod
spec: #maneira com o pod tem que se comportar
    containers: # informações sobre os containeres que irão rodar no pod
        -   name: mongodb
            image: mongo
            ports:
            -   containerPort: 27017
    imagePullSecrets: 
        -   name: <secret-name>

score 0 · Answer 2 · answered Jul 10 '18 at 16:20

0

First, I would double check you are logged into docker at the right registry via cli.

something like docker login <REGISTRY_NAME> -u <CLIENT_ID>

You will want to make sure you have created a k8s secret and bound it to the registry. Maybe check out this post if you haven't already done so. I see your yaml specifies a secret, but is this configured on the registry as well?

answered Jul 10 '18 at 16:20

Lamar

581
7
22

Yes, I successfully logged in with "docker login tlk8s.azurecr.io" and got my secret with "kubectl get secrets" -- result: "acr-secret kubernetes.io/dockerconfigjson 1 6d" – Slothario Jul 10 '18 at 16:32
Just to eliminate issues with the Azure Container Registry and secret configuration I would suggest pushing a generic hello-world style image and create a quick Kubernetes deployment for it. Assuming that loads correctly and you can see the deployment/pods running then you will know the issue is not with your ACR/secrets but instead with your particular container image. (or the opposite if it fails) – KWilson Jul 10 '18 at 16:36
Looks like my hello world image failed, and it's not any faster to test as it's a 8 GB Windows image, unfortunately. Any ideas? – Slothario Jul 10 '18 at 20:26

Azure Container Services: trying and failing to pull image

2 Answers2