Invalid credentials: Google API calendar

Question

I am new to using Google APIs. I followed steps to setup Google Calendar sample code in eclipse using Google client library. I deployed the code to app engine using the cmd command- mvn appengine:update (when tried to deploy through eclipse it gave me an error saying the project is not an App Engine project). Deploying through command line worked and I could launch my web application.

Suddenly, after few API request calls, started getting invalid credentials error:

error:401
domain:global
location: authorization
locationtype : header
message: invalid credentials
reason: auth error

I searched this error and found it may occur if your auth token is invalid or expired. I am not sure what I need to do to get a new auth token. In my case, there is a client_secrets.json file which has the client secrets. I did not put/save or use the auth token anywhere in the code. Following is what I remember when I deployed the code for the first time:

• There was some token which I was asked to paste on the cmd as I was deploying using the command line. After that I could launch my application.
• After that whenever I deployed my code and launched the application, there was no authorization(which happened the first time) and I could use the application and the API subsequently.
• Suddenly I started getting invalid credentials error.

I was using client_secrets.json generated from the developer's console and I did not provide auth token anywhere in the code and I did not save it either. I read that a refresh token can be used and that can avoid expiration of the auth token.

Currently, the issue is that I am getting invalid credentials error and I believe it is because of expiration of auth token. I am not aware about the solution in this case. It is the sample code provided by Google and I believe it automatically handles Oauth authorization (unlike the case where we programmatically call an Oauth url, get a token, store it somewhere and provide it with our subsequent requests). What do I need to do with the sample code so as to resolve this and not to run into the error in future?

Also that the application is running fine locally, but the authorization issue comes when it is deployed on the cloud. Thanks.

Answer 1

The copy/paste token was actually unrelated to the actual credentials being used by the app for accessing the Calendar API; on the command line, it was the App Engine SDK getting permission to modify your App Engine project by uploading the sample app, regardless of what the app itself does.

Now, for the real issue here, it actually seems to be a bug where the GoogleAuthorizationCodeFlow somehow receives an authorization code response which doesn't contain a refreshToken, only an accessToken, and then still goes on to store it in Datastore. Normally, the flow you see is expected to pop up a page saying "This app would like to: ... Have offline access" for the user on the first load, and then your app is supposed to get an accessToken and refreshToken pair, where the accessToken typically expires within 1 hour but then the Credential object knows how to automatically catch the 401 exception and execute the refreshToken to get a new accessToken, all under the hood. This could either be considered a bug in the backend server for returning a credential which lacks a refreshToken, or a bug in the client logic for still assuming there's a refreshToken and thus getting stuck rather than re-issuing a request for access capabilities.

Fortunately, there's an easy workaround. Right now, the fact that you're seeing the 401 error means there's a sticky malformed credential stored in your Datastore and possibly also your Memcache. Navigate to your appengine.google.com page, and assuming you aren't already serving a live production-critical web application there, go to Datastore Viewer on the left, look for the Query -> By kind: drop-down menu, to find StoredCredential, check all the items assuming they likely all came from your Calendar sample, and click "Delete". Also navigate to Memcache Viewer on the left-hand-side menu, and then click Flush Cache.

Now, it appears the reason the backends are trying to return credentials lacking a refreshToken is due to the client trying to incorrectly make use of "auto-approval". This is evidenced by the fact that when I create a brand-new client id with a new JSON client_secrets, then on just the very first time loading the sample calendar app, I find with my logging statements inside the src/main/java/com/google/api/services/samples/calendar/appengine/server/Utils.java file:

Credential credential = newFlow().loadCredential(userId);
if (credential.getRefreshToken() != null) {
  logger.log(Level.SEVERE, "Refresh token is not null");
} else {
  logger.log(Level.SEVERE, "Refresh token is null!");
}

then I do get a refreshToken for each unique login username I try to access it with. However, if I purge the credential from Datastore and Memcache, forcing a re-authentication on a subsequent request, I stop seeing the approval prompt and my credentials stop having refreshTokens, thus causing them to stop working 1 hour later.

Solution (tl;dr)

After purging your Datastore for all entities of kind StoredCredential and flushing your Memcache, simply add .setApprovalPrompt("force") to the newFlow() method inside of src/main/java/com/google/api/services/samples/calendar/appengine/server/Utils.java; the method will look like this:

static GoogleAuthorizationCodeFlow newFlow() throws IOException {
  return new GoogleAuthorizationCodeFlow.Builder(HTTP_TRANSPORT, JSON_FACTORY,
      getClientCredential(), Collections.singleton(CalendarScopes.CALENDAR)).setDataStoreFactory(
      DATA_STORE_FACTORY).setAccessType("offline").setApprovalPrompt("force").build();
}

Optionally, add some logging where the credential is obtained (it's generally bad practice to log the actual accessToken and especially the refreshToken, but during debugging doing it once or twice probably doesn't hurt :), and re-update your app. You'll find that the first time you access your app, you'll get the approval prompt now, and then it should safely work forever since the refreshToken never expires and is now saved in your Datastore.

Note that there's a limit on the number of refreshTokens per client/userId pair ever issued, and if you do purge your Datastore/Memcache during debugging, it effectively leaks a refreshToken. The behavior is just that after 25 such tokens, the earlier ones will automatically be deactivated. See the oauth2 docs for more info on that limit.

Answer 2

In order to receive a refresh token, you need to have access_type=offline in the authorization request: https://developers.google.com/accounts/docs/OAuth2WebServer#offline

Note you will only get the refresh token once, together with the first access token.