Stack Overflow
Stack Overflow

Questions tagged [bq]

bq is a python-based, command-line tool for BigQuery. It allows the creation of datasets, tables, query jobs; and listing and exporting data among other utilities using the command-line.

bq is a python-based, command-line tool for BigQuery. It allows the creation of datasets, tables, query jobs; and listing and exporting data among other utilities using the command-line.

104 questions
1
vote
1 answer

pulling covid deaths from covid19_open_data using bq

I am trying to get COVID19 deaths history per country using Google's SDK. I installed bq (BigQuery) on Ubuntu, but I don't know how to proceed. The online API gives some examples but they seem irrelevant. I expect something like bq extract…
Yuval Harpaz
  • 1,416
  • 1
  • 12
  • 16
1
vote
1 answer

BigQuery: Load avro-files with date column data type as long converted to timestamp

I have troubles getting BigQuery to load timestamps from avro-files correctly. The avro-files have date columns stored as long, with logical type timestamp-micros. As per documentation, BigQuery should store this as timestamp data type. I have also…
gixen
  • 279
  • 4
  • 20
1
vote
2 answers

How to pass multi-word string as a parameter to bq query command in Google BigQuery?

How to pass the multi-word string as a parameter to bq query command in Google BigQuery? My Command: value="param1 param2" bq query \ --use_legacy_sql=false \ --parameter=param_value:string:$value \ "select @param_value" Getting below…
LAHU BHAWAR
  • 93
  • 2
  • 13
1
vote
2 answers

Big Query bq query command hide DML on output?

I want to run a multi-statement query without the statement content being echoed to the stdout. Single-statement queries don't echo statements as can be seen here... bq query --use_legacy_sql=false --project_id=livescore-sandpit-data <
Liam Caffrey
  • 172
  • 1
  • 12
1
vote
2 answers

BiqQuery detected on windows command prompt but not on Bash

I've installed Cloud SDK on my windows 10 machine and I'm able to run commands like "gcloud", "gsutil" and "bq" on my command prompt. However, when I run "gsutil" or "bq" on Bash, this is the error I'm getting. $ bq bash: bq: command not found Then…
johnconnor92
  • 199
  • 1
  • 13
1
vote
0 answers

What is wrong with this Google BigQuery Data Transfer 'bq mk --transfer_config' command's syntax?

I am trying to use the bq CLI to create a BigQuery Data Transfer Service following the instructions here, but for some reason, when I run this in the PowerShell I keep getting this error: The filename, directory name, or volume label syntax is…
AIK DO
  • 288
  • 1
  • 4
  • 13
0
votes
1 answer

When you create an integer-range partitioned table and set the range_partitioning (start, end, interval) value, can you update it later?

Currently I'm working on a task to partition a table that I loaded from CSV. I want to partition on the unix time (INT). If I set the range partitioning (start, end, interval) value now, can I update it later? For example, if I initially set the…
Christine
  • 37
  • 4
0
votes
0 answers

Some Output getting truncated on BigQuery api commands

I am executing BigQuery command via api in CloudShell for extracting view definitions using shell script as shown below for file_name in $(cat view_list.txt) do file_name=${file_name/.sql/} echo "$file_name" query_output=`bq query…
VKarthik
  • 1,379
  • 2
  • 15
  • 30
0
votes
2 answers

How to see BQ query cost before running it?

Previously we had BQ Mate and SuperQuery that are no longer working in new UI to check the cost of query. Is there any free or relatively inexpensive solution that would show cost estimation before running query?
user912830823
  • 1,179
  • 3
  • 14
  • 25
0
votes
0 answers

how to stop bigquery randomly deleting columns if credentials are expired

I have a project which uses BigQuery, however each month I have to rotate the credentials. Sometimes I don't get to that in time. BigQuery will then chew up the database, randomly deleting columns. I have to recreate the tables, adding back 30% of…
dcsan
  • 11,333
  • 15
  • 77
  • 118
0
votes
1 answer

Google BigQuery timestamp comparison

there is a table "tbl1", having columns "name, surname, log_date". "log_date" is timestamp type. How can I retrieve the records logged yesterday? I am struggling with timestamp variable, I did something like: declare yesterday timestamp set…
NewUser
  • 25
  • 1
  • 7
0
votes
1 answer

Retrieve the difference between Strings on BigQuery

Is it possible to retrieve the difference between 2 strings using Bigquery functions. Example. String 1: "Hello world" String 2: "Hello big world" Result expepected: "big" Thanks
Joao Simoes
  • 39
  • 2
0
votes
1 answer

BQ - No longer able to partition table in scheduled queries using run_time

I used to be able to partition my tables in big query using parameters in my table name when setting up a scheduled query by putting my table name and then adding {} brackets to add in the run_time for example my table name would like:…
tl_mus
  • 3
  • 1
0
votes
1 answer

CASE operator in BQ doesn't allow to use different type of arguments

I need to query json strings based on different filters from the same table CASE WHEN provider = 0 THEN JSON_VALUE_ARRAY (raw_data, '$.countries_served') WHEN provider = 1 THEN JSON_VALUE (raw_data, '$.MerchantCountryISOAlpha3') WHEN…
Roman Poluden
  • 1
0
votes
1 answer

Getting an error while Loading BigQuery External table using Bq command line command

I was trying to load a BigQuery external table using Bq load command through Bq command line. Bq load command executed - bq load --source_format=NEWLINE_DELIMITED_JSON {provided dataset_name}.{provided bq external_table_name} gs://{provided…
PyAI
  • 1
  • 3
1 2
3
4 5 6 7