I don't want to generation service accounts for external users I want to share bigquery datasets with- what are the repercussions of this?

Question

I gave a user's Google account access to one of my datasets. They are using this Python script:

def query_stackoverflow():
    client = bigquery.Client()
    query_job = client.query(
        """
        SELECT *
        FROM `myproject.mydata.mytable`
        ORDER BY someColumn DESC
        LIMIT 10"""
    )

    results = query_job.result()

It works but they are seeing this warning:

UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun gcloud auth application-default login and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)

I read through some docs but I don't understand what this means. Does this mean I should put a quota on my project? I know this person and trust them, but does this mean they could use up all of my bq quota with their queries? It also seems like this could be "solved" by using a service account so is this quota a hard limit on non-service account access I can't change?

Giving the user's Google account access is more convenient and secure then creating a service account and generating keys for them.

score 6 · Accepted Answer · answered Sep 02 '20 at 20:54

6

A quota project is used by client libraries, etc. for billing purposes. You can set the quota project using the CLI:

Example command:

cloud auth application-default set-quota-project my-quota-project

gcloud auth application-default set-quota-project

Review BigQuery quotas and limits and implement them. This will improve security and minimize financial risks.

BigQuery Quotas and Limits

There are three primary methods of authorization in Google Cloud. User Credentials created by Google Accounts (Gmail, G Suite, etc.), Service Accounts, and API Keys.

The warning you are receiving is due to Google preferring that applications use Service Accounts for authorization instead of User Credentials. You can disable this warning in your code but I do not recommend that. Instead, create a quota project, create a service account, and lock down the service account to only provide the roles the user requires.

In your question you do not describe how you are using User Credentials to obtain the OAuth Access Token requires to grant API access. If you are passing around tokens, that is not very secure. Service Accounts can be deleted, Service Account Keys can be rotated, etc. providing a better level of security. If you understand Google Cloud, I wrote an article on how to use impersonation. You could use this method to continue using User Credentials to impersonate a Service Account without handing out keys.

Google Cloud – Improving Security with Impersonation

PowerShell – Impersonate Google Service Account

User Account credentials have much lower API calling quotas then Service Accounts. This can cause software to fail that use User Account generated Access Tokens.

answered Sep 02 '20 at 20:54

John Hanley

74,467
6
95
159

I think i was confused, it seems like I should ask the other users' to generate their own service accounts in their own gcp projects and then I share my dataset with _their_ service accounts and then they get billing for big query engine queries right? They are querying my big query datasets but they are paying for the BQ query/compute costs right? – red888 Sep 03 '20 at 16:35
@red888 Adding their service account identity to your Project is a good solution (management/security). However, that will not change who pays the bill. The project is charged and not the users of the project. – John Hanley Sep 03 '20 at 17:48
the project where the service account is, is charged right? they are running big query queries with their service account in their project but accessing my data set right? – red888 Sep 03 '20 at 17:49
@red888 - When you create a service account in Project A and then add that identity to Project B, any resources accessed in Project B are charged to Project B. The identity is not used for billing. There are a few exceptions such as Google Domains, but in Google Cloud, the project who owns the resource is charged and not the users accessing the resource. – John Hanley Sep 03 '20 at 17:52
^ i actually dont think this is true for big query. I think the service account in project A would be using its own project to execute the queries on the tables in project B- after the tables in project A are shared with that service account – red888 Sep 03 '20 at 18:01
2

@red888 - This answer by a BigQuery expert Mikhail Berlyant says you are right about query billing and I am wrong. I was thinking about the resource and not the query.. See if this answer helps: https://stackoverflow.com/a/53952885/8016720 – John Hanley Sep 03 '20 at 18:22
That makes sense. documentation is bad and confusing about this. I actually did need to grant the user access to bigquery.jobs.create in my project and didnt understand why. Its funny because I asked GCP support about that and got a super obtuse and unhelpful response – red888 Sep 03 '20 at 18:28

I don't want to generation service accounts for external users I want to share bigquery datasets with- what are the repercussions of this?

1 Answers1

Linked