6

Where I work we use Google Apps for Work. For the last 9 months we've been using the Gmail API (~2,000 requests per day) to pull in new emails for our support email accounts.

This is how we originally set it up:

  1. Go to https://console.developers.google.com/project/
  2. Click on the project (or create a new one)
  3. Click on API's & Auth
  4. Click on Credentials
  5. Click on Create new Client ID
  6. Click on Service account
  7. Download a JWT (json) for the account.
  8. Follow the node.js quickstart guide with an installed/native type token for the same account, and authorize it through the console. The JWT tokens did not work unless we did this step, once for each account.

We did this for each of our individual support email accounts to avoid having to turn on domain wide delegation for any of them in the admin console. We were then able to authenticate with the tokens using the officially supported npm library googleapis, similar to this:

var google = require('googleapis');

var jwtClient = new google.auth.JWT(
    token.client_email,
    null,
    token.private_key,
    ['https://www.googleapis.com/auth/gmail.readonly'],
    'supportemail@mycompany.com'
);

jwtClient.authorize(function(err, tokens) {
    if (err) {
        return cb(err);
    }

    var gmail = google.gmail('v1');
    var requestOptions = {
        auth: jwtClient,
        userId: 'me',
        id: messageId,
        format: 'raw'
    };

    gmail.users.messages.get(requestOptions, function(err, response) {
        if (err) {
            return cb(err);
        }
        // do stuff with the response
    });
});

Like I said, we used this for a long time and never had any issues. Yesterday around 10am MST every one of the accounts stopped being able to authenticate at the same time, with jwtClient.authorize() suddenly returning the error [Error: unauthorized_client].

I tried doing the same thing with a new token on a new service account (the web interface to get the token has changed quite a bit in the last 9 months), and it returns the same error.

The version of googleapis that we were using was 0.9.7, but we can't get JWT authentication to work on the newest version either.

We opened a ticket with the Google APIs support team, but the support person we spoke with had never read the Gmail API specs before and was ultimately unable to help us, so he redirected us here in order to get in touch with the API engineering support team.

We have noticed that authentication works if we enable the scope for domain wide delegation in the admin console, but we would prefer not to do that. We don't need to impersonate the accounts and would prefer to use an individual JWT for each account.

brismuth
  • 36,149
  • 3
  • 34
  • 37
  • Not affiliated with Google, so just shooting in the dark here :) Could the [expiration recommendations](https://developers.google.com/identity/protocols/OAuth2#expiration) give a clue? `A token might stop working for one of these reasons: The token has not been used for six months`. Could that be it? – Tholle Mar 10 '16 at 23:25
  • 2
    @Tholle, thanks for the suggestion. I looked at the list of reasons, and I don't believe that any are applicable. All of the accounts stopped working at exactly the same time, and we haven't changed the passwords or revoked access, and they are all service accounts, so they shouldn't be limited to 25 refresh tokens. As far as usage, all of the accounts/tokens are used daily. – brismuth Mar 10 '16 at 23:34
  • 1
    I also tried getting new tokens and we get the same error. – brismuth Mar 10 '16 at 23:36
  • 1
    As of right now, with no changes on our side at all, all of our tokens are now *intermittently* working. That leads me to believe that Google is in the process of rolling out a fix. – brismuth Mar 11 '16 at 20:19
  • For each service account, how did you authorize it for a particular support email account? I don't see that happening in the 7 steps you outlined. – Brandon Jewett-Hall Mar 11 '16 at 21:34
  • @BrandonJewett-Hall sorry about that! I completely forgot to put that step, I've added it as step 8. In hindsight, I wonder if the reason step 8 worked for us before was due to a bug, and maybe the problems we started experiencing were due to them fixing it. – brismuth Mar 11 '16 at 21:49
  • For this use case, it seems like you could skip the service account creation and use ordinary 3LO to authorize the client. Then you would save and reuse the refresh token you get, instead of a JWT. – Brandon Jewett-Hall Mar 11 '16 at 22:12
  • @BrandonJewett-Hall Our goal is to give our server a token that will never expire that it can use to authenticate with the gmail API. From what I've read in the docs, that seems to be the exact use case for service accounts. Is that incorrect? – brismuth Mar 11 '16 at 22:19
  • According to the [service account docs](https://developers.google.com/identity/protocols/OAuth2ServiceAccount), service accounts are typically used to access non-user data, but can also access user data if granted domain-wide privileges. The [Installed App auth flow](https://developers.google.com/identity/protocols/OAuth2InstalledApp) sounds like a better fit for your use case. It will produce refresh tokens that are long-lived. – Brandon Jewett-Hall Mar 11 '16 at 22:43
  • If we use the installed app auth flow instead, will we be able to continue getting refresh tokens indefinitely, or is there some limit after which we'll need to reauthorize in a browser? I looked in the docs but couldn't find a straight answer. – brismuth Mar 11 '16 at 23:04
  • Also, our calls are no longer failing at this point. Do you work at Google on the APIs team, and/or do you know if they are planning on restoring the backend change that breaks our current setup with the service accounts? – brismuth Mar 11 '16 at 23:06
  • The installed app flow allows offline (indefinite) refreshing of access tokens. The flow you described isn't a supported flow. As Brandon said, this isn't a typical usage of service accounts. The fact that it worked in the past is more of an accident than a designed feature. – Steve Bazyl Mar 11 '16 at 23:46
  • @SteveBazyl and Brandon thanks for the info. We'll plan on changing our service to use the installed app auth flow instead. – brismuth Mar 11 '16 at 23:55
  • And yes, there was a rollback but it is temporary. – Steve Bazyl Mar 12 '16 at 00:14
  • @SteveBazyl thanks for letting us know, and for rolling back, if that was for us. We'll get our services changed to the new flow ASAP. – brismuth Mar 12 '16 at 00:20
  • @SteveBazyl we've now migrated to the installed app auth flow and are no longer using the hacky JWT auth flow we had before. Thanks again for your suggestions. – brismuth Mar 13 '16 at 07:38

1 Answers1

3

It turns out that the auth flow we were using was never supported, and probably was broken due to a bugfix on Google's part.

In the question comments @Brandon Jewett-Hall and @Steve Bazyl recommended that we use the installed app auth flow instead, as it allows for indefinite refreshing of access tokens and is supported.

More information about the different auth flows can be found in the Google API docs.

Community
  • 1
  • 1
brismuth
  • 36,149
  • 3
  • 34
  • 37