5

I'm trying to write an application that creates mail accounts for thousands of users using the Google Directory API. Creating them one by one works, but is extremely slow. I tried to use the batch requests which is suppose to support up to 1000 requests at once. However with that, only around 50 users are created successfully and the rest of the requests throw 403 errors. If I change the batch size to 40 instead, after the first batch, many requests fail with 5xx errors.

If the batch requests are still limited by the same rate limits, the seem to be worthless as I could just send those requests individually at that slow rate. Is there a better way to do this or is there something else I should do instead?

Jefff
  • 81
  • 4
  • Is your application performing exponential backoff (https://developers.google.com/admin-sdk/directory/v1/limits) to retry the failed operations within the batch? https://developers.google.com/admin-sdk/directory/v1/limits – Jay Lee Oct 28 '13 at 12:01

1 Answers1

2

Batching the requests will certainly save network roundtrips (which can be pretty expensive if you have thousand of users to process). However, the server will still have to execute the request one by one even if it is batched. Take a look at the documentation on Admin SDK

https://developers.google.com/admin-sdk/directory/v1/guides/batch

The special note said: "A set of n requests batched together counts toward your usage limit as n requests, not as one request. The batch request is taken apart into a set of requests before processing."

Emily
  • 1,464
  • 1
  • 9
  • 12
  • Even using exponential backoff, a lot of quotaExceeded error is received. As the Directory API may allow 10 creations / second, I could send a batch with 1,000 creations, wait 100 seconds, and send another one. But, at the first batch of the day, I receive the quota errors. I ended limiting the batch to 12 creations each: this is working without quotaExceeded errors. – dudinha-dedalus May 11 '16 at 15:03