4

I am running microk8s v1.18.8 rev 1609 from 1.18/stable.

Several times I have got my deployments up and running perfectly (as far as I can tell). The images pull from localhost:32000. I have gone through many rounds of updating the deployments and the pods get automatically replaced, with the new images being pulled successfully from the repository.

Then I move onto another project for a few days (having nothing to do with microk8s). I leave microk8s running and untouched. Several times when I've returned to the microk8s project, all the pods have gone away and show an error state (ErrImagePull). If I delete a pod, a new pod tries to replace it, but hangs initially in the ContainerCreating state (last log entry is 'Pulling image "localhost:32000/..."'). Eventually it times out and goes through the ImagePullBackOff and ErrImagePull states. However, the last time I had anything to do with the project, these images were pulling perfectly fine.

I can push the image to localhost:32000 without error. I can pull the image without error. I can pull the image using microk8s.ctr:

microk8s ctr --debug images pull --plain-http localhost:32000/imagename

It works fine. I've tried changing ufw default to allow routed (no effect), iptables -P FORWARD ACCEPT (no effect). microk8s inspect does not report any issues. I've tried microk8s stop followed by microk8s start (no effect). Rebooting my machine (no effect). Everything else about the system appears fine: just the pods trying to pull images fails.

Previously, something in the above made it work again, but not this time. So my main question is "What else can I try?"

My secondary question is: Is this a stable platform for anyone? Can you leave a service/deployment (e.g. an nginx server) running for months without issue? I am tired of leaving a working environment and coming back a little while later to a badly broken system that takes hours/days to fix. I'm having serious doubts about microk8s in particular and k8s in general as a useful platform.

Bradley
  • 53
  • 5
  • So, I gave up trying to fix whatever was wrong and did a microk8s reset. I encountered several microk8s bugs trying to do that (more evidence microk8s is alpha-ware) but eventually succeeded thanks to all the contributors to those bugs. After reset, I recreated all pods and services and all is fine again. Question is for how long? – Bradley Aug 28 '20 at 18:19

1 Answers1

-2

if you pull the image from external registry, if it shows ErrImagePull and ImagePullBackOff error, please try it

kubectl create secret docker-registry regprivate --docker-server=https://privateregistry.com/ --docker-username=user --docker-password=mypassword

spec: imagePullSecrets: - name: regprivate containers: - name: miapp image: privateregistry.com/miapp:v2

PetterTables
  • 1
  • 1