How can I use BigQuery's dataset in Data Lab

Question

I have some datasets in BigQuery, I wonder if there is a way to use the same datasets in Data Lab? As the datasets are big, I can't download it and reload it in Data Lab.

Thank you very much.

Answer 1

The BigQuery Python client library support querying data stored in BigQuery. To load the commands from the client library, paste the following code into the first cell of the notebook:

%load_ext google.cloud.bigquery

%load_ext is one of the many Jupyter built-in magic commands.

The BigQuery client library provides a %%bigquery cell, which runs a SQL query and returns the results as a Pandas DataFrame.

You can query data from a public dataset or from the datasets in your project:

%%bigquery 

SELECT *  

FROM `MY_PROJECT.MY_DATASET.MY_TABLE`  

LIMIT 50 

I was able to successfully get data from the dataset without any issues.

You can follow this tutorial. I hope it helps.