-1

When the total record count is higher (in millions ) then the iterator_1 is set with very minimal value like in a few thousand. But when the data is less, both are having the same set of records. In iterator_1 fetching only the values and then setting iterator. In iterator_2 fetching all records and setting iterator.

Ideally, both are the same right? But getting inconsistent outputs. can someone please help to understand?

TableResult results = bigquery.query(queryConfig);
Iterator<FieldValueList> iterator_1 = results.getValues().iterator();
Iterator<FieldValueList> iterator_2 = results.iterateAll().iterator();
1stenjoydmoment
  • 229
  • 3
  • 14

1 Answers1

2

This is really about the Page<ResourceT> interface which is implemented by TableResult.

Many Google APIs implement list operations (or query operations in this particular case) using pagination. The Page<ResourceT> interface represents a page within this paginated result - an implementation is expected to know how to fetch not only the resources within the current page, but subsequent pages as well.

So if you have a query that returns 25 results, 10 per page, then calling getValues() on the first page will immediately return the first 10 resources, and stop. iterateAll() will return an iterator that makes additional API calls as necessary (but lazily), returning all 25 results before being exhausted.

If you just want to get all the results of the query, use iterateAll(), being aware that it can take a long time and make additional API calls as you iterate. If you want the immediately-available results within the current page, use getValues().

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • Can u pls point me some samples? I want all results to be in Iterator without additional calls. using getValues() to paginate. Not finding any examples in https://github.com/googleapis/java-bigquery/tree/main/samples/snippets/src/main/java/com/example/bigquery . please help. – 1stenjoydmoment Feb 09 '22 at 12:27
  • @1stenjoydmoment: I don't understand what you mean by "I want all results to be in Iterator without additional calls". If the information hasn't already been fetched, how do you expect it to become available without making any additional API calls? (I suspect that you should actually ask a new question at the moment - I've answered your current question, which is about what the difference is. If you need to accomplish a specific task that isn't mentioned in the question, you should ask a new question with that detail.) – Jon Skeet Feb 09 '22 at 12:29
  • Thank You. I just posted new Question. https://stackoverflow.com/questions/71049892/how-to-dd-append-values-to-iteratorfieldvaluelist . please see if u can help. Thanks. – 1stenjoydmoment Feb 09 '22 at 12:46