Gitlab Runner Image with GCP credentials

Question

I am trying to teach my Gitlab Runner image to get custom builder images from my private Docker Registry (GCR running in the Google Cloud).

What did not work out? I created a custom Gitlab Runner image with the ServiceAccount properly set. I started in in non-privileged mode but the wormhole pattern (via docker.sock). On exec-ing into that container (which is based on gitlab/gitlab-runner:v11.3.0) I had to recognise that I cannot do any docker commands in there (neither as root nor as gitlab-user). How the gitlab-runner starts the builder containers afterwards is way above my cognitive capabilities. ;)

# got started via eu.gcr.io/my-project/gitlab-runner:0.0.5 which got taught the GCR credentials

stages:
    - build

build:
    image: docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.2
    stage: build
    script: 
        # only for test if I have access to private docker registry
        - docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.1

What worked out? According to this tutorial you can authenticate via in a before_script block in your .gitlab-ci.yml files. That worked out.

# got started via gitlab/gitlab-runner:v11.3.0 

stages:
    - build

before_script:
    - apk add --update curl python which bash
    - curl -sSL https://sdk.cloud.google.com | bash
    - export PATH="$PATH:/root/google-cloud-sdk/bin"
    - gcloud components install docker-credential-gcr
    - gcloud auth activate-service-account --key-file=/key.json
    - gcloud auth configure-docker --quiet

build:
    image: docker:18.03.1-ce
    stage: build
        # only for test if I have access to private docker registry
        - docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.1

The Question This means that I have to do this (install gcloud & authenticate) in each build run - I would prefer to have done this in the gitlab-runner image. Do you have an idea how to achieve this?

Please show your `.gitlab-ci.yml`. There seems to be a bit of confusion here between the gitlab-ci-multi-runner and the actual images you run builds on. — Jakub Kania, Sep 24 '18 at 11:43
You seem to have two problems here. Gitlab runner authenticating to gcloud to pull image used for jobs, authenticating inside jobs. It seems you are able to run the jobs on images you prepared so the first one works, now what is the docker image definition for the image you try to run the job on? For both images ideally. — Jakub Kania, Sep 27 '18 at 15:25

score 4 · Answer 1 · answered Sep 28 '18 at 07:17

Finally I found a way to get this done.

Teach the vanilla gitlab-runner how to pull from your private GCR Docker Repo

GCP

Create Service account with no permissions in IAM & Admin
Download Json Key
Add Permissions in Storage Browser
1. Select bucket holding your images (eg eu.artifacts.my-project.appspot.com)
2. Grant permission Storage Object Admin to the service account

Local Docker Container

Launch a library/docker container and exec into it (with Docker Wormhole Pattern docker.sock volume mount)
Login into GCR via (Check the url of your repo, in my case its located in Europe, therefore the eu prefix in the url) docker login -u _json_key --password-stdin https://eu.gcr.io < /etc/gitlab-runner/<MY_KEY>.json
Verify if it works via some docker pull <MY_GCR_IMAGE>
Copy the content of ~/.docker/config.json

Gitlab config.toml configuration

Add the following into your config.toml file [[runners]] environment = ["DOCKER_AUTH_CONFIG={ \"auths\": { \"myregistryurl.com:port\": { \"auth\": \"<TOKEN-FROM-DOCKER-CONFIG-FILE>\" } } }"]

Vanilla Gitlab Runner Container

Run the runner eg like this docker run -it \ --name gitlab-runner \ --rm \ -v <FOLDER-CONTAININNG-GITLAB-RUNNER-CONFIG-FILE>:/etc/gitlab-runner:ro \ -v /var/run/docker.sock:/var/run/docker.sock \ gitlab/gitlab-runner:v11.3.0

Your .gitlab-ci.yml file

Verify the done work via a .gitlab-ci.yml
1. Use an image which is located in your private GCP Container Registry

Teach your builder images how to push to your private GCR Docker Repo

GCP

Add permissions to your service account
1. Grant permission Storage Legacy Bucket Reader to your service account in the Storage Browser

Custom Docker Builder Image

Add your Service Account key file to your your custom image FROM docker:18.03.1-ce ADD key.json /<MY_KEY>.json

Your .gitlab-ci.yml file

Add the following script into your before_script section docker login -u _json_key --password-stdin https://eu.gcr.io < /key.json

Final Thoughts

Now the vanilla gitlab-runner can pull your custom images from your private GCR Docker Repo. Furthermore those pullable custom images are also capable of talking to your private GCR Docker Repo and eg push the resulting images of your build pipeline.

That was quite complicated stuff. Maybe Gitlab enhances the support for this usecase in the future.

score 0 · Answer 2 · answered May 02 '22 at 14:09

This example config worked for me in values.yaml:

config: |
[[runners]]
  [runners.docker]
    image = "google/cloud-sdk:alpine"
  [runners.kubernetes]
    namespace = "{{.Release.Namespace}}"
    image = "google/cloud-sdk:alpine"
    [runners.cache]
      Type = "gcs"
      Path = "runner"
      Shared = true
      [runners.cache.gcs]
        BucketName = "runners-cache"
    [[runners.kubernetes.volumes.secret]]
      name = "service-account-credentials"
      mount_path = "keys"
      read_only = true

Where service-account-credentials is a secret containing credentials.json

then in .gitlab-ci.yml you can do:

gcloud auth activate-service-account --key-file=/keys/credentials.json

Hope it helps

Louis Baumann · Answer 3 · 2018-09-25T11:14:53.160

have you tried to use google cloudbuild? i had the same problem and solved it like this:

echo ${GCR_AUTH_KEY} > key.json
gcloud auth activate-service-account --key-file key.json
gcloud auth configure-docker

gcloud builds submit . --config=cloudbuild.yaml --substitutions _CI_PROJECT_NAME=$CI_PROJECT_NAME,_CI_COMMIT_TAG=${CI_COMMIT_TAG},_CI_PROJECT_NAMESPACE=${CI_PROJECT_NAMESPACE}

cloudbuild.yaml:

steps:
 - name: gcr.io/cloud-builders/docker
    id: builder
    args: 
      - 'build'
      - '-t'
      - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
      - '.'
 - name: gcr.io/cloud-builders/docker
    id: tag-runner-image
    args: 
      - 'tag'
      - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
      - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:latest'
images:
 - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
 - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:latest'

just use google/cloud-sdk:alpine as image in the gitlab-ci stage

Yes I do have GCB pipelines running for a different project - they are awesome. But in this project I cannot use GCB due to there is no integration yet for Gitlab Repos. — Hubert Ströbitzer, Sep 25 '18 at 11:13
you don't need to have a integration in gcb. put the cloudbuild.yaml in your project, use the image google/cloud-sdk:alpine in your stage and execute the gcloud commands as shown above. the variable GCR_AUTH_KEY should be stored as a variable in gitlab. — Louis Baumann, Sep 26 '18 at 05:48

Gitlab Runner Image with GCP credentials

3 Answers3

Teach the vanilla gitlab-runner how to pull from your private GCR Docker Repo

Teach your builder images how to push to your private GCR Docker Repo

Final Thoughts

Linked