3

There is a scheduled notebook, that uses BigQuery client and service account with Owner rights. When I run the cells manually, it makes an update to BQ table. There is one project for both BQ and Vertex AI.

I've found a similar question, but there is no output in bucket folder: Google Cloud Vertex AI Notebook Scheduled Runs Aren't Running Code?

In schedules section this notebook is stuck on Initializing: The endlesly initializing notebook is 'a_test_marketplace_update_1650393238920'

Here's the notebook: First part of notebook

Second part of notebook

Update: I've tried to schedule cells one by one, and all of the stuck attempts cannot get through BigQuery:

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'dialogflow-293713-f89fd8f4ed2d.json'

bigquery_client = bigquery.Client()

QUERY = f"""
INSERT `dialogflow-293713.chats.Ежедневная сводка маркетплейса` (date, effectiveness, operatorWorkload)
VALUES({period}, {effectiveness}, {redirectedToSales}, {operatorWorkload})
"""

Query_Results = bigquery_client.query(QUERY)
Helen Kapatsa
  • 61
  • 9

1 Answers1

1

This way of authorization worked!

from google.cloud import bigquery
from google.oauth2 import service_account
import json


raw_credential = { "dictionary. copy the dict elements of your credential.json file" }

service_account_info = json.loads(json.dumps(raw_credential))
credentials = service_account.Credentials.from_service_account_info(service_account_info)

client = bigquery.Client(credentials=credentials)
query = """ Your Query """
df = client.query(query).to_dataframe()
#see some results. remove if its not needed.
print(df.head())

# OPTIONAL: If you want to move data to a google cloud storage bucket
from google.cloud import storage

client = storage.Client()
bucket_name = 'my-bucket-id'
bucket = client.get_bucket(bucket_name)
# if folder `output` does not exist it will be created. You can use the name as you want.
bucket.blob("output/output.csv").upload_from_string(df.to_csv(), 'text/csv')

Resolved on Issue Tracker in this thread.

Helen Kapatsa
  • 61
  • 9