How to implement Exponential Backoff for Google Gmail API?

Question

I am currently using Gmail API to send emails on user's behalf. The Mails are sent one by one and the average size of recipients is 500. I frequently see { "code" : 500, "errors" : [ { "domain" : "global", "message" : "Backend Error", "reason" : "backendError" } ], "message" : "Backend Error" }

as well as some occurrences of

{ "code" : 429, "errors" : [ { "domain" : "usageLimits", "message" : "Rate Limit Exceeded", "reason" : "rateLimitExceeded" } ], "message" : "Rate Limit Exceeded" }

Google has suggested implementing Exponential backoff strategy to resolve these errors. I have implemented below solution, but it doesn't seem to work and is not helping with these errors.Here is my implementation;

public  GoogleCredential createCredentialWithRefreshToken(String accessToken, String refreshToken) 
    {
           GoogleCredential credential =  new GoogleCredential.Builder().setTransport(new NetHttpTransport())
                .setJsonFactory(new JacksonFactory())
                .setClientSecrets(Constants.GOOGLE_CLIENT_ID, Constants.GOOGLE_CLIENT_SECRET)
                .setRequestInitializer(setHttpTimeout())
                .build();
            credential.setAccessToken(accessToken).setRefreshToken(refreshToken);

           return credential;

    }
public HttpRequestInitializer setHttpTimeout() {
          return new HttpRequestInitializer() {

            @Override
            public void initialize(HttpRequest httpRequest) throws IOException {
              httpRequest.setUnsuccessfulResponseHandler(new HttpBackOffUnsuccessfulResponseHandler(backOff()));
              httpRequest.setConnectTimeout(3 * 60000);  // 3 minutes connect timeout
              httpRequest.setReadTimeout(3 * 60000);  // 3 minutes read timeout
            }

            private final ExponentialBackOff.Builder BACK_OFF = new ExponentialBackOff.Builder().setInitialIntervalMillis(500);

            private BackOff backOff() {
               return BACK_OFF.build();
            }   

          };
     }
public static Gmail getGmailServiceForGoogleAccount(GoogleAccount googleAcct){
         Gmail gmailService = null;
        GoogleCredential credential = new Utils().createCredentialWithRefreshToken(googleAcct.getAccess_token(),googleAcct.getRefresh_token());
        gmailService  =   new Gmail.Builder(new NetHttpTransport(),
                                new JacksonFactory(), credential)
                                .setApplicationName("test")
                                .build();

        return gmailService;
     }

What is wrong with this implementation? Am i implementing the custom HttpRequestInitializer correctly. Where could i set the log statements to find out if a request is being retried as per Exponential policy?

Please suggest

score 1 · Answer 1 · answered Oct 06 '20 at 19:40

I see this is an old question, but will leave my answer here in case anyone finds it useful.

The problem with the code is that it is calling .setRequestInitializer() on the GoogleCredential.Builder, which sets the initializer for token requests and not the service API requests.

See the documentation here

Sets the HTTP request initializer for refresh token requests to the token server or null for none.

Instead the initializer should be configured on the Google service client and you can chain it with the Credential response handler to preserve its functionality too.

Something like this should work for the provided example:

    public static HttpRequestInitializer requestInitializer(Credential credential) {
        return new HttpRequestInitializer() {

            @Override
            public void initialize(HttpRequest httpRequest) throws IOException {
                httpRequest.setConnectTimeout(3 * 60000);  // 3 minutes connect timeout
                httpRequest.setReadTimeout(3 * 60000);  // 3 minutes read timeout
                // chain response handler with the handler from the credential
                // that handles retries for authentication errors
                HttpUnsuccessfulResponseHandler responseHandler =
                        new HttpBackOffUnsuccessfulResponseHandler(backOff());
                httpRequest.setUnsuccessfulResponseHandler((req, res, retry) ->
                        credential.handleResponse(req, res, retry)
                                || responseHandler.handleResponse(req, res, retry));
            }

            private final ExponentialBackOff.Builder BACK_OFF = new ExponentialBackOff.Builder().setInitialIntervalMillis(500);

            private BackOff backOff() {
                return BACK_OFF.build();
            }

        };
    }
    public static Gmail getGmailServiceForGoogleAccount(GoogleAccount googleAcct){
        GoogleCredential credential = new Utils().createCredentialWithRefreshToken(googleAcct.getAccess_token(),googleAcct.getRefresh_token());
        return new Gmail.Builder(new NetHttpTransport(), new JacksonFactory(), requestInitializer(credential))
                .setApplicationName("test")
                .build();
    }

score 0 · Answer 2 · answered Jul 02 '17 at 06:08

0

Check the Exponential Backoff for Java implementation:

  ExponentialBackOff backoff = new ExponentialBackOff.Builder()
        .setInitialIntervalMillis(500)
        .setMaxElapsedTimeMillis(900000)
        .setMaxIntervalMillis(6000)
        .setMultiplier(1.5)
        .setRandomizationFactor(0.5)
        .build();
    request.setUnsuccessfulResponseHandler(new HttpBackOffUnsuccessfulResponseHandler(backoff));

Check this SO post for additional reference.

answered Jul 02 '17 at 06:08

ReyAnthonyRenacia

17,219
5
37
56

i have mentioned the Exponential Backoff implementation above in my code. Do you see any issue with that implementation!. Should i customise more params? – zee Jul 05 '17 at 13:11

How to implement Exponential Backoff for Google Gmail API?

2 Answers2