Google cloudbuild secrets not substituted

Question

I am trying to retrieve secrets from the secrets manager in the cloudbuild.yaml file but I can't find a way.

- name: 'gcr.io/cloud-builders/gcloud'
    args:
      - beta
      - run
      - deploy
      - ${REPO_NAME}
      - --region=europe-west2
      - --image=gcr.io/$PROJECT_ID/${REPO_NAME}:$COMMIT_SHA
      - --service-account=${_SERVICE_ACCOUNT}
      - --cpu=2
      - --allow-unauthenticated
      - --set-env-vars=GCP_DB_INSTANCE_NAME=$$GCP_DB_INSTANCE_NAME
      - --set-env-vars=PG_DATABASE=$$PG_DATABASE
      - --set-env-vars=PG_PASSWORD=$$PG_PASSWORD
      - --set-env-vars=PG_USER=$$PG_USER
      - --set-env-vars=GCP_PROJECT=$$GCP_PROJECT
      - --set-env-vars=GCP_BUCKET_NAME=$$GCP_BUCKET_NAME
      - --add-cloudsql-instances=$$GCP_DB_INSTANCE_NAME
    secretEnv: [ 'GCP_DB_INSTANCE_NAME', 'PG_DATABASE', 'PG_PASSWORD', 'PG_USER', 'GCP_PROJECT', 'GCP_BUCKET_NAME' ]
availableSecrets:
  secretManager:
    - versionName: projects/$PROJECT_ID/secrets/GCP_DB_INSTANCE_NAME/versions/latest
      env: GCP_DB_INSTANCE_NAME
    - versionName: projects/$PROJECT_ID/secrets/PG_DATABASE/versions/latest
      env: PG_DATABASE
    - versionName: projects/$PROJECT_ID/secrets/PG_PASSWORD/versions/latest
      env: PG_PASSWORD
    - versionName: projects/$PROJECT_ID/secrets/PG_USER/versions/latest
      env: PG_USER
    - versionName: projects/$PROJECT_ID/secrets/GCP_PROJECT/versions/latest
      env: GCP_PROJECT
    - versionName: projects/$PROJECT_ID/secrets/GCP_BUCKET_NAME/versions/latest
      env: GCP_BUCKET_NAME

But the variables are not substituted. I have logged the values in my api and that is what I get:

2021-08-05T22:31:33.437926Z key value PG_DATABASE $PG_DATABASE
2021-08-05T22:31:33.437965Z key value PG_USER $PG_USER
2021-08-05T22:31:33.437985Z key value PG_PASSWORD $PG_PASSWORD
2021-08-05T22:31:33.438063Z key value GCP_PROJECT $GCP_PROJECT
2021-08-05T22:31:33.438093Z key value GCP_BUCKET_NAME $GCP_BUCKET_NAME

How can I substitute the secrets in my step?

How are you logging the values? Why not use the native Cloud Run Secret Manager integration instead? — sethvargo, Aug 05 '21 at 22:58
I am logging these value in my nodejs api. Something like `console.log('key value', process.env. PG_DATABASE)`. What is the Cloud Run Secret Manager integration? — Kevin Amiranoff, Aug 05 '21 at 23:04

score 6 · Accepted Answer · answered Aug 06 '21 at 14:36

Instead of injecting these variables at build time, it would be better to inject them at runtime. As written, the secrets will be viewable in plaintext by anyone with permission to view the Cloud Run service. That's because they are resolved during the build step and set as environment variables. Furthermore, if you were to revoke or change one of these secrets, the Cloud Run service would continue to operate with the old value.

A better solution is to use the native Cloud Run Secret Manager integration, which resolves secrets at instance boot. It would look like this:

- name: 'gcr.io/cloud-builders/gcloud'
  args:
    - run
    - deploy
    - ${REPO_NAME}
    - --region=europe-west2
    - --image=gcr.io/$PROJECT_ID/${REPO_NAME}:$COMMIT_SHA
    - --service-account=${_SERVICE_ACCOUNT}
    - --cpu=2
    - --allow-unauthenticated
    - --set-secrets=GCP_DB_INSTANCE_NAME=projects/$PROJECT_ID/secrets/GCP_DB_INSTANCE_NAME:latest,PG_DATABASE=projects/$PROJECT_ID/secrets/PG_DATABASE:latest // continue
    - --add-cloudsql-instances=$$GCP_DB_INSTANCE_NAME

Cloud Run will automatically resolve the secrets when it boots a new instance. You'd need to grant $SERVICE_ACCOUNT permissions to access the secret.

Awesome this worked for me, but for some reason I had to use project number but not projectid, here is my code `--set-secrets=DB_USER=projects/$PROJECT_NUMBER/secrets/DB_USER/versions/latest,DB_PASSWORD=projects/$PROJECT_NUMBER/secrets/DB_PASSWORD/versions/latest,DB_DATABASE=projects/$PROJECT_NUMBER/secrets/DB_DATABASE/versions/latest,DB_HOST=projects/$PROJECT_NUMBER/secrets/DB_HOST/versions/latest,DB_PORT=projects/$PROJECT_NUMBER/secrets/DB_PORT/versions/latest` — Umamaheswararao Meka, May 25 '23 at 14:07

score 1 · Answer 2 · answered Aug 05 '21 at 23:01

Can you please give it a try as below?

- name: 'gcr.io/cloud-builders/gcloud'
  secretEnv: [ 'GCP_DB_INSTANCE_NAME', 'PG_DATABASE', 'PG_PASSWORD', 'PG_USER', 'GCP_PROJECT', 'GCP_BUCKET_NAME' ]
  entrypoint: 'bash'
  args:
  - -c
  - |
    gcloud beta run deploy ${REPO_NAME} --region=europe-west2 --image=gcr.io/$PROJECT_ID/${REPO_NAME}:$COMMIT_SHA --service-account=${_SERVICE_ACCOUNT} --cpu=2 --allow-unauthenticated --set-env-vars=GCP_DB_INSTANCE_NAME=$$GCP_DB_INSTANCE_NAME --set-env-vars=PG_DATABASE=$$PG_DATABASE --set-env-vars=PG_PASSWORD=$$PG_PASSWORD --set-env-vars=PG_USER=$$PG_USER --set-env-vars=GCP_PROJECT=$$GCP_PROJECT --set-env-vars=GCP_BUCKET_NAME=$$GCP_BUCKET_NAME --add-cloudsql-instances=$$GCP_DB_INSTANCE_NAME

According to the docs, you have to specify the flag -c in the args field so anything after it will be treated as a command. Ref: https://cloud.google.com/build/docs/securing-builds/use-secrets

Google cloudbuild secrets not substituted

2 Answers2

Linked