6

I'm using the RemoteAPI (Java) to go through a large dataset, ~90K entities, and perform some data migration.

int CHUNK_SIZE = 500;
int LIMIT = 900; 

QueryResultList<Entity> result = ds.prepare(entityQuery)
.asQueryResultList(
    FetchOptions.Builder
    .withPrefetchSize(CHUNK_SIZE)
    .limit(LIMIT)
    .chunkSize(CHUNK_SIZE)
).startCursor(cursor);

With the query LIMIT set to 900the result.size() is the entire dataset, ~90K, instead of 900. If I try a lower LIMIT, say 300, the result size is the expected one (300).

What am I missing here? From the documentation I couldn't figure out why it produces the behaviour I'm describing here.

Xenolion
  • 12,035
  • 7
  • 33
  • 48
Nuno Rosa
  • 177
  • 1
  • 8
  • 1
    I'm maybe telling something stupid but have you tried changing the order of your options? FetchOptions.Builder.withLimit(LIMIT) .withPrefetchSize(CHUNK_SIZE) .chunkSize(CHUNK_SIZE) ).startCursor(cursor); – yannicksoldati Apr 04 '16 at 08:29
  • Correct me if I'm wrong but I don't see the `.limit` method to be in the Method Summary for the `FetchOptions.Builder` object (see: https://cloud.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/FetchOptions.Builder). Should be `.withLimit` instead. – Héctor Valverde Apr 06 '16 at 10:20

1 Answers1

0

Based on these examples (http://www.programcreek.com/java-api-examples/index.php?api=com.google.appengine.api.datastore.QueryResultList)

I think that you should use .withLimit(LIMIT) instead of .limit(LIMIT) within the .asQueryResultList options

So I would restructure your code as follows:

FetchOptions options = FetchOptions.Builder
    .withLimit(LIMIT)
    .withPrefetchSize(CHUNK_SIZE)
    .chunkSize(CHUNK_SIZE);

QueryResultList<Entity> result = ds.prepare(entityQuery)
    .asQueryResultList(options);

Then get cursor

result.getCursor();
nardeas
  • 633
  • 6
  • 14