1

How does one pass a secret from Google Secrets Manager (GSM) to a Cloud Function when using Cloud Build? The below cloudbuild.yaml has three steps. Further, I'm using volumes to create permanent storage between build steps. I can confirm GSM retrieval by Cloud Build. However, when I attempt to pass a secret in yaml format using --env-vars-file I encounter the following error ...

Already have image (with digest): gcr.io/cloud-builders/gcloud
ERROR: gcloud crashed (AttributeError): 'str' object has no attribute 'items'

cloudbuild.yaml:

steps:
  - name: 'gcr.io/cloud-builders/gcloud'
    volumes:
    - name: 'secrets'
      path: '/secrets'
    entrypoint: "bash"
    args:
      - "-c"
      - |
        echo -n 'gsm_secret:' > /secrets/my-secret-file.txt
  - name: 'gcr.io/cloud-builders/gcloud'
    volumes:
    - name: 'secrets'
      path: '/secrets'
    entrypoint: "bash"
    args:
      - "-c"
      - |
        gcloud components update
        gcloud beta secrets versions access --secret=MySecret latest >> /secrets/my-secret-file.txt
        cat /secrets/my-secret-file.txt
  - name: 'gcr.io/cloud-builders/gcloud'
    volumes:
      - name: 'secrets'
        path: '/secrets'
    args: [
      'functions', 'deploy', 'gsm-foobar',
      '--project=[...]',
      '--trigger-http',
      '--runtime=go111',
      '--region=us-central1',
      '--memory=256MB',
      '--timeout=540',
      '--entry-point=GSM',
      '--allow-unauthenticated',
      '--source=https://source.developers.google.com/[...]',
      '--service-account', '[...]@appspot.gserviceaccount.com',
      '--env-vars-file', '/secrets/my-secret-file.txt'
    ]

Update: Usage of volumes is not required as /workspace is permanent storage between steps in Cloud Build. Also, gcloud components update is no longer necessary as the default Cloud SDK version, as of today, is 279.0.0

A Solution:

steps:
  - name: 'gcr.io/cloud-builders/gcloud'
    entrypoint: "bash"
    args:
      - "-c"
      - |
        echo "gsm_secret: $(gcloud beta secrets versions access --secret=MySecret latest)" > /workspace/my-secret-file.txt
        cat /workspace/my-secret-file.txt
  - name: 'gcr.io/cloud-builders/gcloud'
    args: [
      'functions', 'deploy', 'gsm-foobar',
      [...]
      '--entry-point=GSM',
      '--allow-unauthenticated',
      '--source=https://source.developers.google.com/[...]',
      '--service-account', '[...]@appspot.gserviceaccount.com',
      '--env-vars-file=/workspace/my-secret-file.txt'
    ]
JamesB
  • 21
  • 4
  • I understand that you put your secret in plain text as function env var? If you do so, your secret is no longer secret, you can see your decrypted env var into Cloud Functions console. keep your secrets secret as long as you can and decrypt them only at runtime in your function if you want to achieve the higher level of security. – guillaume blaquiere Feb 11 '20 at 08:23
  • The alternative would be accessing GSM secrets (`secretmanagerpb.AccessSecretVersionRequest`) on every function invocation. Or levering `init()` so that a cold start invocation would retrieve a GSM secret and store in a global scope variable thus caching it between function invocations. – JamesB Feb 11 '20 at 18:47
  • yes, and this alternative is better, even if the cold start is slightly longer. In memory secret storage is safer than in env var metadata. – guillaume blaquiere Feb 11 '20 at 20:27
  • Does this answer your question? [How to specify secretEnv to cloudbuild.yaml via gcloud cli args or environment variables](https://stackoverflow.com/questions/60002762/how-to-specify-secretenv-to-cloudbuild-yaml-via-gcloud-cli-args-or-environment-v) – sethvargo Mar 06 '20 at 17:26

2 Answers2

2

As of 2021 February 10, you can access Secret Manager secrets directly from Cloud Build using the availableSecrets field:

steps:
- id: 'deploy'
  name: 'gcr.io/cloud-builders/gcloud'
  entrypoint: 'bash'
  args:
  - '-c'
  - 'gcloud functions deploy --set-env-vars=SECRET=$$MY_SECRET'
  secretEnv: ['MY_SECRET']
availableSecrets:
  secretManager:
  - versionName: 'projects/my-project/secrets/my-secret/versions/latest'
    env: 'MY_SECRET'

Documentation

sethvargo
  • 26,739
  • 10
  • 86
  • 156
  • This seems a bit redundant if you also can just access [`--set-secret`](https://cloud.google.com/sdk/gcloud/reference/run/deploy#--set-secrets) nowadays at the `deploy` command. – luukvhoudt Dec 08 '21 at 20:19
1

On second read, I realize your 2nd step puts the secret value in the file. I think you're missing the newline.

NB I've not tried this for myself!

Ensure you have a newline at the end of your secrets file.

See: https://cloud.google.com/functions/docs/env-var

Update: tried it ;-)

I think your issue was the final newline.

Using the following in a step prior to the deployment, works:

echo "gsm_secret: $(gcloud beta secrets versions access --secret=MySecret latest)" > /secrets/my-secret-file.txt

Or, more simply, perhaps:

steps:
  - name: "gcr.io/cloud-builders/gcloud"
    entrypoint: /bin/bash
    args:
      - "-c"
      - |
        gcloud functions deploy ... \
        --set-env-vars=NAME=$(gcloud beta secrets versions access --secret=name latest)

Also, see secretEnv. This is a more elegant mechanism..This functionality should perhaps be augmented by Google to support secret manager (in addition to KMS).

Community
  • 1
  • 1
DazWilkin
  • 32,823
  • 5
  • 47
  • 88