4

gitlab-runner executor failing on Microk8s Raspberry pi cluster - ContainersNotInitialized: "containers with incomplete status: [init-permissions]"

Any help or just suggestions as to how to troubleshoot this further would be appreciated!

I am trying to us the gitlab-runner executor on a mincrok8s raspberry pi cluster. I am getting the following errors:

Running with gitlab-runner 14.5.2 (e91107dd)
  on gitlab-runner-gitlab-runner-5779968774-dppmf kxEK3YoP
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab
Using Kubernetes executor with image arm64v7/ubuntu:20.04 ...
Using attach strategy to execute scripts...
Preparing environment
Waiting for pod gitlab/runner-kxek3yop-project-5-concurrent-0cp2v8 to be running, status is Pending
    ContainersNotInitialized: "containers with incomplete status: [init-permissions]"
    ContainersNotReady: "containers with unready status: [build helper]"
    ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod gitlab/runner-kxek3yop-project-5-concurrent-0cp2v8 to be running, status is Pending
    ContainersNotInitialized: "containers with incomplete status: [init-permissions]"
    ContainersNotReady: "containers with unready status: [build helper]"
    ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod gitlab/runner-kxek3yop-project-5-concurrent-0cp2v8 to be running, status is Pending
    ContainersNotInitialized: "containers with incomplete status: [init-permissions]"
    ContainersNotReady: "containers with unready status: [build helper]"
    ContainersNotReady: "containers with unready status: [build helper]"
ERROR: Job failed (system failure): prepare environment: waiting for pod running: pod status is failed. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

I installed the gitlab-runner executor with helm. Here is the values.yaml I used:

gitlabUrl: http://<my-url-is-here-you-not-need-it>/

runnerRegistrationToken: "xxxxxxxxxxxxxxxx"

concurrent: 20

checkInterval: 30

# For RBAC support:
rbac:
  create: false

runners:
  image: ubuntu:18.04
  helpers:
    image: gitlab/gitlab-runner-helper:arm64-latest

  privileged: false

  builds:
    # cpuLimit: 200m
    # memoryLimit: 256Mi
    cpuRequests: 100m
    memoryRequests: 128Mi

  services:
    # cpuLimit: 200m
    # memoryLimit: 256Mi
    cpuRequests: 100m
    memoryRequests: 128Mi

  helpers:
    # cpuLimit: 200m
    # memoryLimit: 256Mi
    cpuRequests: 100m
    memoryRequests: 128Mi
KNOX
  • 367
  • 1
  • 3
  • 15
  • Did you see the link in the error: https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading? RaspberryPi has a default .bash_logout file in your home directory. Does it work if you get rid of the file? Let me know, I'm interested in setting up a k8s cluster of gitlab runners on Raspberry Pis – cvanbeek Jan 07 '22 at 15:18

1 Answers1

1

I had the same issue today and it was beacause of the default helper image which is basically wrong in the chart. Something went really wrong with gitlab's helper image registry because all of the arm images look to be built for amd64! Look here for example: https://hub.docker.com/r/gitlab/gitlab-runner-helper/tags?page=1&name=arm

In any case here's what fixed my problem, in my values.yaml this is how I specified the runners section:

runners:
  tags: "ascalia-k8s"
  secret: gitlab-runner-secret
  namespace: gitlab
  config: |
    [[runners]]
      name = "Kubernetes Prod Runner"
      executor = "kubernetes"
    [runners.kubernetes]
      image = "ubuntu:20.04"
      helper_image = "gitlab/gitlab-runner-helper:arm64-1278d3da"
Erokos
  • 106
  • 1
  • 6