Python gdata API returning empty data with oauth2

Question

A while back I wrote a GAE application that scrapes some information from a google spreadsheet using the gdata-python-client library. Everything has been working fine until recently (in the last week) when Google finally removed the ClientLogin method. They now only allow oauth2 for authentication. This has completely broken my application, and I'm having a heck of a time getting oauth2 to work.

I went into the application's admin console, created a service account in the Credentials manager, and downloaded the json data configuration. I then implemented the authentication like this:

    path = os.path.join(os.path.split(__file__)[0],'api-auth.json')
    auth_data = json.load(open(path))

    path = os.path.join(os.path.split(__file__)[0],'key.txt')
    private_key = open(path).read()

    credentials = SignedJwtAssertionCredentials(
      auth_data['client_email'],
      private_key,
      scope=(
        "https://www.googleapis.com/auth/drive",
        "https://spreadsheets.google.com/feeds",
        "https://docs.google.com/feeds"
      ),
      sub = '<app admin email>')
    http = httplib2.Http()
    http = credentials.authorize(http)
    auth2token = gdata.gauth.OAuth2TokenFromCredentials(credentials)

    gd_client = gdata.spreadsheets.client.SpreadsheetsClient()
    gd_client = auth2token.authorize(gd_client)

    # Open the main roster feed
    roster_sheet_key = '<key from spreadsheet url>'
    feed = gd_client.GetWorksheets(roster_sheet_key)

api-auth.json contains some of the fields from the json data downloaded from Google. key.txt contains the private key from the downloaded data, with the \n text replaced with actual newlines.

There aren't any failures with it logging in as far as I can tell. The problem I'm having is when it calls GetWorksheets(). That call throws a parsing error:

  File "/Users/tim/Desktop/projects/testing/rostermgmt.py", line 184, in get
    feed = gd_client.GetWorksheets(roster_sheet_key)
  File "/Users/tim/Desktop/projects/testing/gdata/spreadsheets/client.py", line 108, in get_worksheets
    **kwargs)
  File "/Users/tim/Desktop/projects/testing/gdata/client.py", line 640, in get_feed
    **kwargs)
  File "/Users/tim/Desktop/projects/testing/gdata/client.py", line 278, in request
    version=get_xml_version(self.api_version))
  File "/Users/tim/Desktop/projects/testing/atom/core.py", line 520, in parse
    tree = ElementTree.fromstring(xml_string)
  File "<string>", line 125, in XML
  ParseError: no element found: line 1, column 0

I dug into the code for the gdata library a bit, and it appears that the http request for the data returns a blank string. I also dug into the oauth2client library and it seems like it's not properly making http request for the oauth token. One issue here is that there looks like there's a few different ways to do this and no one good example from Google as the "official" method. My original implemntation is based on Using OAuth2 with service account on gdata in python, but it's clearly not working for me.

Answer 1

I sorted this out. There were a few things I had to do differently:

1. Download the private key from Google as a p12 file. Convert the .p12 file I downloaded from Google into a pem file with the following command. The default password is 'notasecret'.

   openssl pkcs12 -in key.p12 -nodes -nocerts > privatekey.pem
   

2. Force authentication to occur by using the Google apiclient library. The final code is below. client_email is the email address provided for the service user in the GAE console.

   import httplib2
   from oauth2client.client import SignedJwtAssertionCredentials
   from apiclient.discovery import build
   import gdata.spreadsheets
   import gdata.spreadsheets.client
   import gdata.gauth
   import os
   
   path = os.path.join(os.path.split(__file__)[0],'privatekey.pem')
   with open(path) as keyfile:
     private_key = keyfile.read()
   
   credentials = SignedJwtAssertionCredentials(
     '<client email>',
     private_key,
     scope=(
       'https://www.googleapis.com/auth/drive',
       'https://spreadsheets.google.com/feeds',
       'https://docs.google.com/feeds',
     ))
   http_auth = credentials.authorize(httplib2.Http())
   authclient = build('oauth2','v2',http=http_auth)
   
   auth2token = gdata.gauth.OAuth2TokenFromCredentials(credentials)
   
   gd_client = gdata.spreadsheets.client.SpreadsheetsClient()
   gd_client = auth2token.authorize(gd_client)
   
   # Open the main roster feed
   roster_sheet_key = '<key from spreadsheet url>'
   feed = gd_client.GetWorksheets(roster_sheet_key)
   

I unfortunately couldn't figure out how to use the SpreadsheetService API that I had been using previously, but SpreadsheetsClient works well enough. I don't notice any significant performance difference between the two for what I'm doing with it.