0

I am connecting to BigQuery tables using dbplyr and the tables are partitioned, usually by date or time (the column is called _PARTITIONDATE or _PARTITIONTIME). In BigQuery, you can filter using the WHERE _PARTITIONDATE > "some date" to access segments of the table and improve query performance.

But when I connect to a table via dbplyr, there is no partition column on which to filter. Is this expected behavior?

mundos
  • 459
  • 6
  • 14

1 Answers1

1

I found that you can use the build_sql function in dbplyr to filter on the partition column for the table and then continue to write your query using dbplyr e.g:

tbl(con, "table_name") %>%
filter(!!build_sql("DATE(_PARTITIONTIME > '2021-06-21'", con = con)

Whenever there is no dbplyr equivalent, you can do this.

mundos
  • 459
  • 6
  • 14