1

I am trying to run a Python script to access Secrets Manager in GCP, but I keep running into the error above: "403 Request had insufficient authentication scopes".

I have been following these instructions in the GCP docs. I can successfully access a secret version with the command line using

gcloud secrets versions access {{ version-id }} --secret={{ "secret-id" }}

but cannot access a secret version using the Python function from the docs

def access_secret_version(project_id, secret_id, version_id):
    """
    Access the payload for the given secret version if one exists. The version
    can be a version number as a string (e.g. "5") or an alias (e.g. "latest").
    """

    # Import the Secret Manager client library.
    from google.cloud import secretmanager

    # Create the Secret Manager client.
    client = secretmanager.SecretManagerServiceClient()

    # Build the resource name of the secret version.
    name = f"projects/{project_id}/secrets/{secret_id}/versions/{version_id}"

    # Access the secret version.
    response = client.access_secret_version(request={"name": name})

    # Print the secret payload.
    # snippet is showing how to access the secret material.
    payload = response.payload.data.decode("UTF-8")

    return payload

What am I missing here?

The IAM permissions for my account contain "Secrets Manager Secret Accessor." I don't think that is the issue since I can access a secret version from the command line. But the Python script keeps failing.

Any ideas what the issue might be?

italo
  • 11
  • 3
  • 3
    Where are you running this code from? The error message might indicate that this is executed from a GCE instance with insufficient access scopes. If this is the case, make sure the instance's service account has the right permissions, but also that the instance access scopes is set to "cloud-platform" (a.k.a. "Allow full access to all Cloud APIs"). See https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances for reference. – Thomas Laporte Jul 28 '21 at 08:37
  • Thanks for the reply. I am trying to run code a colleague of mine has already built which I've clone from a GitHub repo. My Python code exists locally and I am using VSCode/CL to edit and run the script. – italo Jul 28 '21 at 19:32

2 Answers2

1

@thomas-laporte is correct.

It's confusing but, when you run gcloud (CLI) commands, you access account credentials one way and, when you run SDK code locally, you (need to) access credentials a different way. This explains why the gcloud command works but your code does not; they're accessing credentials differently.

It's not the ideal way but, if you gcloud auth application-default login you can configure Google's Application Default Credentials1 to authenticate your code.

Run that command then try your code again, it should work.

Application Default Credentials are available to all Google's SDKs and simplify the auth process.

It is preferred that you create a Service Account and, if running off-GCP, that you generate a key for the account and then reference the location of the key with GOOGLE_APPLICATION_CREDENTIALS to run your code.

But, for a quick hit, it's fine to gcloud auth application-default login.

1 - Google used to have better documentation explaining Application Default Credentials but, this appears to have gone (and replaced with Finding credentials automatically. John Hanley has a very clear and comprehensive explanation:

https://www.jhanley.com/google-cloud-application-default-credentials/

DazWilkin
  • 32,823
  • 5
  • 47
  • 88
  • Thank you for the reply. My code exists locally and I am using VSCode to edit/run it. In the end, I ended up using a Service Account by changing line 11 of the Python function to `client = secretmanager.SecretManagerServiceClient.from_service_account_json('/path/to/key.json')` But, I am curious as to why using my User Account wasn't working. I will give your suggestion a try and see if that will do the trick. I'm still fairly new to GCP and was wondering if you had any resources detailing when it is preferable to use a Service Account vs User Account? – italo Jul 28 '21 at 19:46
  • A very good question. In a nutshell, Service Accounts should be used (to provide an identity) to code|programs. A former name for Service Accounts was "robots" and this made the intent more obvious. User accounts must (!) be used to authenticate a human user. User accounts use so-called 3-legged OAuth (the user is prompted with a dialog outlining permissions) and the user gets to decide whether they want the service to e.g. access their Gmail account, write to their Google Drive. Service Accounts use 2-legged OAuth which doesn't provide the prompt and is unable to access user info directly. – DazWilkin Jul 28 '21 at 19:56
  • Your solution is slightly more difficult than it needs to be. Application Default Credentials means that, if you set (must `export`) the environment variable `export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json`) you can then run your code without having to `.from_service_account(....)`, you can just `client = secretmanager.SecretManagerServiceClient()`. See: https://cloud.google.com/secret-manager/docs/reference/libraries#client-libraries-usage-python. – DazWilkin Jul 28 '21 at 19:59
  • Not only does this simplify your code but it makes it easier to move it to e.g. Compute Engine, Cloud Run etc. because the credentials are provided by these services (automagically) for you (without need to `GOOGLE_APPLICATION_CREDENTIALS=..` and without any other change). – DazWilkin Jul 28 '21 at 19:59
  • Lastly, you may wonder: "What if I want to run a program and it needs to access user data?"? In this case, the program should still run as a Service Account (because it's a program) but then it must authenticate the user's (User) account and the user will be prompted to delegate (their) authority to the program to operate on their behalf.... Hey Italo's Awesome Program would like to send an email on your behalf, permission to access your Gmail?" – DazWilkin Jul 28 '21 at 20:02
  • Thank you so much for the detailed response! Your help is greatly appreciated. – italo Jul 28 '21 at 20:23
  • You're very welcome! Please consider tagging my answer as an answer if it helped. – DazWilkin Jul 28 '21 at 20:35
  • If you're developing locally as a human, you should authenticate as a human with `gcloud auth login --update-adc`. This will be detected as Application Default Credentials (ADC) by all the client libraries and use your user credential. This means you need to grant permissions to your human account in IAM. Please don't download a service account key - it is not necessary and decreases your security. – sethvargo Jul 28 '21 at 20:39
  • Re: "Google used to have better documentation explaining Application Default Credentials", please see https://cloud.google.com/docs/authentication/best-practices-applications – sethvargo Jul 28 '21 at 20:42
  • @sethvargo that's more like the one I was looking for. It doesn't appear in the top results for "Application Default Credentials" though...maybe have a word ;-) – DazWilkin Jul 28 '21 at 22:50
  • @DazWilkin for a search engine company, we are quite terrible at SEO :( – sethvargo Jul 29 '21 at 14:29
0

If you're running code on your local machine, you should install the Google Cloud SDK and authenticate to the SDK as your personal user account:

gcloud auth login --update-adc

This will open a browser window. Select your user account and follow the prompts. Upon success, your credentials will be saved to gcloud.

If you're using an official Google Cloud client library, it will automatically detect the Cloud SDK and use those credentials.

sethvargo
  • 26,739
  • 10
  • 86
  • 156