Questions tagged [pybigquery]
23 questions
0
votes
0 answers
BigQuery mismatch columns
I have multiple in .CSV files in Google cloud storage but I am unable to push data into table because number of columns in .CSV files are different.
Let's say I have 10 .CSV files with columns A, B, C, D, and I have another 10 .CSV files with…
kakaji
- 161
- 9
0
votes
1 answer
BigQuery bulk update existing table rows from json file after adding new columns to the table
I have a bigQuery table which has around 2M rows which are loaded from a JSON file. Actual fields in JSON file are 10 but table has 7 columns as per initial DDL. Now I altered the table and added remaining three columns. After altering, the values…
code tutorial
- 554
- 1
- 5
- 17
0
votes
1 answer
Loading Pub/Sub Message to BigQuery with Python
I am tryting to pull Pub/Sub message and load to message to BigQuery with python. I can pull the messages but I could not be able to load it to BigQuery. Here is a code example that write. Do you know how to load this message to BigQuery by using…
Enes Doğukan Özgener
- 13
- 4
0
votes
1 answer
Big query error - Scalar subquery produced more than one result
select
CASE LOWER(channel)
WHEN 'sem' THEN
CASE
WHEN (LOWER(SPLIT(campaign_name,'_')[SAFE_OFFSET(9)]) = 'us') THEN 'us independents'
WHEN (LOWER(SPLIT(campaign_name,'_')[SAFE_OFFSET(9)]) = 'intl') THEN 'intl…
harsha k
- 1
- 1
0
votes
1 answer
SQLAlchemy: Is there a way to append raw query to query object?
I'm trying to create a custom query with BigQuery in Google
I can do almost everything using SQLAlchemy filter function
query = TestModel.query.filter(TestModel.timestamp.__gt__(1010101010))
But I want to add custom filter on where clause like…
Eric Lee
- 700
- 2
- 9
- 30
0
votes
1 answer
Data streaming in Apache superset from BQ?
I am new to superset and wanted to know if there's any way to perform data streaming in big query using apache superset? Currently, I have set up the database in apache superset with big query but when I update the table data using SQL commands in…
Shivam Sahil
- 4,055
- 3
- 31
- 62
-1
votes
1 answer
Schema update options should only be specified with WRITE_APPEND disposition, or with WRITE_TRUNCATE disposition on a table partition
I have a cloud functions that loads a dataframe to bigquery and want to allow field addtions
My job_config is as follow but I always get an error saying that the ALLOW FIELD ADDTION
is only available with table partittions. The destination table is…
Aleksei Díaz
- 35
- 5
-2
votes
1 answer
BigQuery Client Url
My server is a closed-network and i want to make a whitelist only for bigquery client and gsutil.
I have tried adding following url
https://cloud.google.com/bigquery
https://console.cloud.google.com/
But it turns out bigquery needs a little…
ariq hadi
- 3
- 1