0

I want to pass some more options than just QueryParameters (this is already possible) to a bigquery job I am executing with pydatalab through a Query Object Instance.

You can find available options in the docs for the API: https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs

Maybe someone of you is more into that topic and can tell me how, for example, pass the defaultDataset.

Guillem Xercavins
  • 6,938
  • 1
  • 16
  • 35
Nico Albers
  • 1,556
  • 1
  • 15
  • 32
  • I'm afraid it's not implemented in the library and there is no easy workaround. Have you considered filing a feature request in the [issue tracker](https://github.com/googledatalab/pydatalab/issues)? – Guillem Xercavins Mar 02 '18 at 09:27

1 Answers1

1

If you are using pydatalab, then you should take a look at the Python module called google.datalab.bigquery. This module allows you, amongs other things, to initializes an instance of a Query object with different parameters (e.g.: sql query, context, sources, etc...).

You also have a class for the DatasetName.

Not exactly sure why you need to set the default dataset. What you can do is something like:

my_obj = bq.DatasetName('myProjectId', 'myDatasetId')

and then you can get the datasetId: my_obj.dataset_id.

For instance:

import google.datalab.bigquery as bq
my_obj = bq.DatasetName('projectid', 'datasetid')
query = bq.Query("SELECT * FROM " + "`" + my_obj.dataset_id + ".table`" + " LIMIT 1000")

output_options = bq.QueryOutput.table(use_cache=False)
result = query.execute(output_options=output_options).result()
result
Nico Albers
  • 1,556
  • 1
  • 15
  • 32
Alvaro
  • 813
  • 8
  • 16
  • Yeah, I'm already using it. I was not able to pass other options, especially not the defaultDatasetId – Nico Albers Feb 24 '18 at 11:06
  • Understand Nico. You don't have the same options than using the API. I'll edit my answer to provide more details. – Alvaro Feb 24 '18 at 15:11
  • I know that I per default don't have the same options. But maybe somebody is aware of how to do that without requesting the API manually or easily patch that method – Nico Albers Feb 24 '18 at 15:12
  • Thanks for editing. I do **not** want to manually format the query. I want to use some bypass options like defaultDatasetId (this is an example, not the question, one use case of passing defaultDataset would be to write queries for a production/staging system and execute them automatically on the right one, but keep them copy-pastable to the web ui). – Nico Albers Feb 24 '18 at 16:01