0

I'm using a python script within a google cloud function and recently made a very minor update to the script. In testing however, the python script is failing to write data into bigquery.

Below are the libraries I'm using

google-cloud-storage==1.18.0
numpy==1.17.0
dask[dataframe]==2.1.0
pandas==0.25.0
gcsfs==0.2.3
google.auth==1.6.3
google-api-python-client==1.7.11
toolz==0.10.0
fsspec==0.4.1
google-cloud-bigquery[pandas,pyarrow]==1.7.0

Below is my code

def cloudfunction(data, context):  
    
    from google.cloud import storage
    from google.cloud import bigquery
    import pandas as pd
    import dask.dataframe as dd
    import io
    import numpy as np
    import datetime as dt
    from googleapiclient import discovery
    from pandas.io.json import json_normalize
    import google.auth
    import math
    
    # Confirming Oauth # 
    bigquery_client = bigquery.Client()
    
    #Big Query Destination #
    dataset_ref = bigquery_client.dataset('my_dataset')
    table_ref = dataset_ref.table('my_datatable')
    job_config = bigquery.LoadJobConfig()
    job_config.write_disposition = bigquery.WriteDisposition.WRITE_TRUNCATE
    
    df = pd.DataFrame({'account-start': ['2000-01-01', '2000-01-03', '2000-01-02'],
                       'db-id': [1, 2, 3],
                       'name': ['Alice', 'Bob', 'Charlie'}) 
        
    bigquery_client.load_table_from_dataframe(df, table_ref,job_config=job_config).result()

And here is the error

Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 449, in run_background_function _function_handler.invoke_user_function(event_object) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 268, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 265, in call_user_function event_context.Context(**request_or_event.context)) File "/user_code/main.py", line 596, in cloudfunction bigquery_client.load_table_from_dataframe(df, table_ref,job_config=job_config).result(): File "/env/local/lib/python3.7/site-packages/google/api_core/page_iterator.py", line 212, in _items_iter for page in self._page_iter(increment=False): File "/env/local/lib/python3.7/site-packages/google/api_core/page_iterator.py", line 243, in _page_iter page = self._next_page() File "/env/local/lib/python3.7/site-packages/google/api_core/page_iterator.py", line 369, in _next_page response = self._get_next_page_response() File "/env/local/lib/python3.7/site-packages/google/api_core/page_iterator.py", line 419, in _get_next_page_response method=self._HTTP_METHOD, path=self.path, query_params=params File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 479, in api_request timeout=timeout, File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 337, in _make_request method, url, headers, data, target_object, timeout=timeout File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 374, in _do_request return self.http.request( File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 157, in http return self._client._http File "/env/local/lib/python3.7/site-packages/google/cloud/client.py", line 187, in _http self._http_internal.configure_mtls_channel(self._client_cert_source) AttributeError: 'AuthorizedSession' object has no attribute 'configure_mtls_channel'

Any help would be most appreciated in solving this problem

jwlon81
  • 339
  • 3
  • 15

1 Answers1

1

Looks like you have the same issue, as in this other case.

So, including google-cloud-core 1.5.0 might solve the problem.

Jordi
  • 127
  • 1
  • 4