4

I searched a lot how to authenticate/authorize Google's client libraries and it seems no one agrees how to do it.

Some people states that I should create a service account, create a key out from it and give that key to each developer that wants to act as this service account. I hate this solution because it leaks the identity of the service account to multiple person.

Others mentioned that you simply log in with the Cloud SDK and ADC (Application Default Credentials) by doing:

$ gcloud auth application-default login

Then, libraries like google-cloud-storage will load credentials tied to my user from the ADC. It's better but still not good for me as this require to go to IAM and give every developer (or a group) the permissions required for the application to run. Moreover, if the developer runs many applications locally for testing purposes (e.g. microservices), the list of permissions required will probably be very long. Also it will be hard to understand why we gave such permissions after some time.

The last approach I encountered is service account impersonation. This resolve the fact of exposing private keys to developers, and let us define the permission required by an application, let's say A once, associate them to a service account and say:

Hey, let Julien act as the service account used for application A.

Here's a snippet of how to impersonate a principal:

from google.auth import impersonated_credentials
from google.auth import default

from google.cloud import storage

target_scopes = ['https://www.googleapis.com/auth/devstorage.read_only']

credentials, project = default(scopes=target_scopes)

final_credentials = impersonated_credentials.Credentials(
    source_credentials=credentials,
    target_principal="foo@bar-271618.iam.gserviceaccount.com",
    target_scopes=target_scopes
)

client = storage.Client(credentials=final_credentials)

print(next(client.list_buckets()))

If you want to try this yourself, you need to create the service account you want to impersonate (here foo@bar-271618.iam.gserviceaccount.com) and grant your user the role Service Account Token Creator from the service account permission tab.

My only concern is that it would require me to wrap all Google Cloud client libraries I want to use with something that checks if I am running my app locally:

from google.auth import impersonated_credentials
from google.auth import default

from google.cloud import storage

target_scopes = ['https://www.googleapis.com/auth/devstorage.read_only']

credentials, project = default(scopes=target_scopes)
if env := os.getenv("RUNNING_ENVIRONMENT") == "local":
    credentials = impersonated_credentials.Credentials(
        source_credentials=credentials,
        target_principal=os.environ["TARGET_PRINCIPAL"],
        target_scopes=target_scopes
    )

client = storage.Client(credentials=credentials)
print(next(client.list_buckets()))

Also, I have to define the scopes (I think it's the oauth2 access scopes?) I am using, which is pretty annoying.

My question is: I am I going the right direction? Do I overthink all of that? Is there any easier way to achieve all of that?

Here's some of the source I used:

Update 1

This topic is discussed here.

I've made a first proposition here to support this enhancement.

Update 2

The feature has been implemented! See here for details.

MadJlzz
  • 767
  • 2
  • 13
  • 35
  • I think I am hearing you say that you need credentials for developers. What I think will be useful to understanding your story is to describe how the final application will be used? Are you building an application where an end user will need credentials or are you building an app that will eventually run as one identity? – Kolban Oct 01 '21 at 23:33
  • I see. Imagine a Cloud Run app using a service account A that read files from a bucket. How can I develop and test my application locally without exposing myself to the problems mentioned above? – MadJlzz Oct 01 '21 at 23:42
  • I would then assume that when you deploy your Cloud Run App that you would configure it to "run as" a desired service account at deployment time. This would mean that your actual code would have no need to name or retrieve keys for an explicit service account. If that is the case, then your developers should indeed test with application-default credentials. Your developers should gcloud authenticate with sufficient permissions to write to "A" bucket. Does it have to be your production bucket? I'd say no ... pass in a bucket name as a variable and use a development bucket. – Kolban Oct 02 '21 at 04:28
  • This is what I was doing until now but... Permissions maintenance is super hard. Let's say you're deploying 10 different applications within Cloud Run and each application have strictly different permissions. Developing on those apps means that you have to give a principal (user or group) all the permissions required. Looking at the IAM, you will see that people or groups have a bunch of roles attached to them and you won't be able to know why they are needed and if you can remove some of them if the requirements of an app changes. Problem that won't occur if you impersonate service accounts? – MadJlzz Oct 02 '21 at 13:55
  • @Kolban I assume that the final application will be a production called "Fake Spacestation". See: https://www.space.com/russian-film-crew-soyuz-docks-at-space-station – WJA Oct 07 '21 at 10:56

1 Answers1

5

You can use a new gcloud feature and impersonate your local credential like that:

gcloud auth application-default login --impersonate-service-account=<SA email>

It's a new feature. Being a Java and Golang developer, I checked and tested the Java client library and it already supports this authentication mode. However, it's not yet the case in Go. And I submitted a pull request to add it into the go client library.

I quickly checked in Python, and it seems implemented. Have a try on one of the latest versions (released after August 3rd 2021) and let me know!!

Note: A few is aware of your use case. I'm happy not to be alone in this case :)

guillaume blaquiere
  • 66,369
  • 2
  • 47
  • 76
  • Perfect! I tried that but it doesn't seems to be entirely supported yet: `google.auth.exceptions.DefaultCredentialsError: The file C:\Users\klaer\AppData\Roaming\gcloud\application_default_credentials.json does not have a valid type. Type is impersonated_service_account, expected one of ('authorized_user', 'service_account', 'external_account').` At least I am not alone in this case indeed! – MadJlzz Oct 02 '21 at 14:12
  • @MadJlzz Feel free to open a topic in GitHub, as we can see there are many others that can benefit with this feature. [https://github.com/googleapis/google-auth-library-python/issues] – Bryan L Oct 04 '21 at 08:03
  • 1
    @BryanL I guess the related issue is here: https://github.com/googleapis/google-auth-library-python/issues/762 I will comment back by providing details. – MadJlzz Oct 07 '21 at 10:05