0

I have a project which uses BigQuery, however each month I have to rotate the credentials. Sometimes I don't get to that in time. BigQuery will then chew up the database, randomly deleting columns.

I have to recreate the tables, adding back 30% of the columns that were nuked.

I think this is maybe because of creds caching across nodes / distributed nature, some lag between credentials, meaning some fields get deleted or the query gets mangled somehow. It's a repeated pattern every couple of months.

Is there a reliable way to test if I have write authorization across a whole cluster of BQ nodes (or however it's working under the hood)?

I was thinking of doing some type of test write to a completely different table, but if there is this creds issue it's not certain that will work reliably either. It's also fairly hard to repro as it only seems to happen on the edge case as the creds are going bad. I've noticed this with many gcp services that you'll get intermittent failures/successes as your creds expire, due to creds caching/IDK what.

Code is something like the below:

from google.oauth2 import service_account
from google.cloud import bigquery

bq_creds = service_account.Credentials.from_service_account_file(
    base_config.read('DEFAULT_CREDS_PATH'))

bq_client = bigquery.Client(credentials=bq_creds, project=bq_creds.project_id)

job = bq_client.query(qstring)
result = job.result()

While this is a huge pain for me, and we lose some metrics data and have to rebuild the tables, it could be catastrophic for other projects.

dcsan
  • 11,333
  • 15
  • 77
  • 118
  • Can you tell me if there are any other services or products you are using with BigQuery? – kiran mathew Jun 29 '22 at 13:32
  • yes like DataStudio, but that's not related to this – dcsan Jul 04 '22 at 02:37
  • It seems that this issue has to be investigated further. If you have a support plan, I would recommend you to raise a ticket with Google Cloud Support. If not, create a thread in Google’s Issue Tracker using this [template](https://issuetracker.google.com/issues/new?component=187149&template=1162659). You can also STAR the issue to receive automatic updates and give it traction by referring to this [link](https://developers.google.com/issue-tracker/guides/subscribe#starring_an_issue). – kiran mathew Jul 08 '22 at 13:19

0 Answers0