Python IO module unknown encoding while unittesting GAE

Question

I have written some unittests for my Python Google App Engine app. Below is a distillation of the problematic code.

class TestCase(unittest.TestCase):

    def setUp(self):
        from google.appengine.ext import testbed
        self.testbed = testbed.Testbed()
        self.testbed.activate()
        self.testbed.init_urlfetch_stub()
        self.testbed.init_blobstore_stub()
        ndb.get_context().clear_cache()

    def tearDown(self):
        self.testbed.deactivate()

    def testing(self):

        from google.cloud import storage
        client = storage.Client()

I am getting an encoding LookupError while opening a file (my gcloud application credentials) using the io library.

Here is the relevant stack trace and offending io code (from pytest):

self = <test_1.TestCase testMethod=testing>

    def testing(self):
        from google.cloud import storage

>       client = storage.Client()

/Users/alex/projects/don/don_server/mobile/tests/test_1.py:66: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Users/alex/projects/don/don_server/lib/google/cloud/storage/client.py:59: in __init__
    _http=_http)
/Users/alex/projects/don/don_server/lib/google/cloud/client.py:223: in __init__
    _ClientProjectMixin.__init__(self, project=project)
/Users/alex/projects/don/don_server/lib/google/cloud/client.py:177: in __init__
    project = self._determine_default(project)
/Users/alex/projects/don/don_server/lib/google/cloud/client.py:190: in _determine_default
    return _determine_default_project(project)
/Users/alex/projects/don/don_server/lib/google/cloud/_helpers.py:181: in _determine_default_project
    _, project = google.auth.default()
/Users/alex/projects/don/don_server/lib/google/auth/_default.py:277: in default
    credentials, project_id = checker()
/Users/alex/projects/don/don_server/lib/google/auth/_default.py:117: in _get_gcloud_sdk_credentials
    credentials_filename)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

filename = '/Users/name/.config/gcloud/application_default_credentials.json'

    def _load_credentials_from_file(filename):
        """Loads credentials from a file.

        The credentials file must be a service account key or stored authorized
        user credentials.

        Args:
            filename (str): The full path to the credentials file.

        Returns:
            Tuple[google.auth.credentials.Credentials, Optional[str]]: Loaded
                credentials and the project ID. Authorized user credentials do not
                have the project ID information.

        Raises:
            google.auth.exceptions.DefaultCredentialsError: if the file is in the
                wrong format.
        """
>       with io.open(filename, 'r') as file_obj:
E       LookupError: unknown encoding:

I don't get this error in when I run this code on my GAE local development server. Furthermore when I open the credentials file using a shell (I checked the file attribute on the io module and it's the same) no error is raised.

score 0 · Accepted Answer · answered Aug 01 '17 at 17:42

For some reason the correct environment variables are not set on a Mac when running the unittests through pycharm. Adding the following code (from this answer) solved it for me:

import os
os.environ['LC_ALL'] = 'en_US.UTF-8'
os.environ['LANG'] = 'en_US.UTF-8'

Python IO module unknown encoding while unittesting GAE

1 Answers1