0

I am trying to pull and run the images which are of a size more than 1 GB in Kubernetes pods. After 2 minutes, pods getting destroyed and recreating and the same issue is happening in a loop. Below is what I am getting while running ///kubectl get pods 

NAME                                                             READY   STATUS              RESTARTS   AGE
dynamic-agent-fe7b9f8e-bce6-471b-a6b9-26cc52b02bba-4nxcd-psngk   0/3     ContainerCreating   0          <invalid>


Events:
  Type    Reason     Age   From                      Message
  ----    ------     ----  ----                      -------
  Normal  Scheduled  92s   default-scheduler         Successfully assigned default/dynamic-agent-fe7b9f8e-bce6-471b-a6b9-26cc52b02bba-4nxcd-qvg8c to k8sworker1-test
  Normal  Pulling    41s   kubelet, k8sworker1-test  pulling image "docker-all.xxx.xxx.net/sap/ppiper/node-browsers:v3"

After the above step, it continues to pull for 120 seconds after that k8s killing the pod and recreating it again same pulling process repeats...

Kapil Khandelwal
  • 1,096
  • 12
  • 19
Senthilraj Chettykumar
  • 51
  • 2
  • 6
  • add kubelet logs from the node where the pod is scheduled – Arghya Sadhu Apr 02 '20 at 04:35
  • If you can add more information, this would help as well. Did you try to download the image locally? Is there a proxy in front of the registry, maybe there's a misconfiguration. What is the error message you get, while downloading it locally. Is the image public available, that someone of us can test it? Let us know, what you already tried. – Manuel Apr 02 '20 at 07:29
  • If you can reduce the image size that will help things a lot; 1 GB is an informal practical upper bound on the size of an image, or else you'll run into things like the timeout issues you're encountering. If you have a full C toolchain (`apt-get install build-essential`) rebuilding it using a multi-stage Dockerfile can be a helpful approach. – David Maze Apr 02 '20 at 11:30
  • Hi sorry for the late reply.. actuall these pods are running in test environment cluster however if I am changing to prod cluster the pods are not running without any issue. So I spot it is a capacity issue, but however I could see there is enough diskspace is available in kubernetes test environment cluster I am not sure what parameter is overloading when the containers are pulled – Senthilraj Chettykumar Apr 09 '20 at 11:11
  • @DavidMaze I already informed the team to use less weight images, but they want to scale up the infrastructure.. I am not sure which parameter I have to check to scale the capacity in test environment cluster where these pods are running fine in prod environment cluster I will post the kubectl describe nodes logs here.. – Senthilraj Chettykumar Apr 09 '20 at 11:13
  • can you provide logs from journalctl -u kubelet ? – Vit Apr 22 '20 at 13:44

0 Answers0