3

I have the following problem:

We have a table (partitioned by timestamp) that saves data from IoT devices (a lot of data, the expected ratio for new incoming data is about every 30sec each device).

The problem is that we will serving a query service and users can set a "since" and "until" filters over a "ts" field (the timestamp), but I want to paginate the results by 10,20,... (wherever the user sets "pageSize").

How I can do this? since saving the whole result in a temp table (and using Tabledata.list API) is not an option 'cause I would have a temp table for each different request (different in terms of filters...)

Thanks in advance!!!

sarasa
  • 75
  • 2
  • 11

1 Answers1

0

For Nodejs the library automatically paginates using the token from previous request for you as described here https://cloud.google.com/bigquery/docs/paging-results#bigquery-paging-nodejs

on the other hand you have some options for manual pagination if you read the implementation code here: https://github.com/googleapis/nodejs-bigquery/blob/master/src/index.ts

Pentium10
  • 204,586
  • 122
  • 423
  • 502
  • 1
    Hi, I saw the first link and I dropped because I can't filter the data, there is no way to pass a query. I will just pull all rows, unless I missing something... And regarding the other link, yes I read the code and I'm doing the following but it keeps replying with the same 10 first rows: const queryOptions: Query = { maxResults: 10, pageToken, query } const queryOptions2: QueryOptions = { autoPaginate: true, maxResults: 10, } return await bigquery.query(queryOptions, queryOptions2) – sarasa Jan 31 '19 at 16:26