3

This question is still unsolved! Please answer if you know

Bug

I have filed a bug here

http://code.google.com/p/google-api-python-client/issues/detail?id=131&thanks=131&ts=1335708962

While working on my gdrive-cli project, I ran into this error attempting to upload a UTF-8 markdown file, using the "text/plain" mime-type. I also tried with "text/plain;charset=utf-8" and got the same result.

Here is the stacktrace I got:

Traceback (most recent call last):
  File "./gdrive-cli", line 155, in <module>
    handle_args(args)
  File "./gdrive-cli", line 92, in handle_args
    handle_insert(args.insert)
  File "./gdrive-cli", line 126, in handle_insert
    filename)
  File "/home/tom/Github/gdrive-cli/gdrive/gdrive.py", line 146, in insert_file
    media_body=media_body).execute()
  File "/usr/local/lib/python2.7/dist-packages/apiclient/http.py", line 393, in execute
    headers=self.headers)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 401, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1544, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1294, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1231, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 955, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 989, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 951, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 809, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4518: ordinal not in range(128)

And the command I had to issue to generate it was:

gdrive-cli --insert README.md "readme file" none "text/plain" README.md

You can get the exact README.md file at the time this problem occurred here, http://tomdignan.com/files/README.md

The relevant code from the SDK examples follows. The parameters going in are in order:

a service instance, "README.md", "readme file", None (python keyword), "text/plain", and "README.md"

def insert_file(service, title, description, parent_id, mime_type, filename):
    """Insert new file.

    Args:
        service: Drive API service instance.
        title: Title of the file to insert, including the extension.
        description: Description of the file to insert.
        parent_id: Parent folder's ID.
        mime_type: MIME type of the file to insert.
        filename: Filename of the file to insert.
    Returns:
        Inserted file metadata if successful, None otherwise.
    """
    media_body = MediaFileUpload(filename, mimetype=mime_type)
    body = {
        'title': title,
        'description': description,
        'mimeType': mime_type
    }

    # Set the parent folder.
    if parent_id:
        body['parentsCollection'] = [{'id': parent_id}]


    try:
        file = service.files().insert(
                body=body,
                media_body=media_body).execute()

        # Uncomment the following line to print the File ID
        # print 'File ID: %s' % file['id']

        return file
    except errors.HttpError, error:
        print "TRACEBACK"
        print traceback.format_exc()
        print 'An error occured: %s' % error
        return None
bossylobster
  • 9,993
  • 1
  • 42
  • 61
Thomas Dignan
  • 7,052
  • 3
  • 40
  • 48
  • One bug is enough. If you truly believe this is a bug, which it seems you do, then we will respond to the bug you filed. Posting here just duplicates the number of responses that need to be given to you. – Vic Fryzel Apr 30 '12 at 02:58
  • @VicFryzel I thought about it and you are right. But the question is stuck. Tried to delete and failed. If it makes any difference, I posted the question before the bug. – Thomas Dignan Apr 30 '12 at 12:08

2 Answers2

6

Traceback:

...
resp = pool.request(method, page, fields = fields)
  File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/request.py", line 79, in request
**urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/request.py", line 139, in request_encode_body
**urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/connectionpool.py", line 415, in urlopen
body=body, headers=headers)
  File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/connectionpool.py", line 267, in _make_request
conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python2.7/httplib.py", line 958, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 992, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 812, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 181: ordinal not in range(128)

urllib3

filepost.py:

def encode_multipart_formdata(fields, boundary=None):
    ...
    body = BytesIO()
    ...
    return body.getvalue(), content_type

request.py:

def request(self, method, url, fields=None, headers=None, **urlopen_kw):
    ...
    else:
        return self.request_encode_body(method, url, fields=fields,
                                        headers=headers,
                                        **urlopen_kw)

def request_encode_body(self, method, url, fields=None, headers=None,
                            encode_multipart=True, multipart_boundary=None,
                            **urlopen_kw):
    ...
    if encode_multipart:
        body, content_type = encode_multipart_formdata(fields or {},
                                                       boundary=multipart_boundary)
    ...
    headers = headers or {}
    headers.update({'Content-Type': content_type})

    return self.urlopen(method, url, body=body, headers=headers,
                        **urlopen_kw)

httplib

httplib.py:

def _send_request(self, method, url, body, headers):
    ...
    if body and ('content-length' not in header_names):
        self._set_content_length(body)
    ...

def _set_content_length(self, body):
    # Set the content-length based on the body.
    thelen = None
    try:
        thelen = str(len(body))
    except TypeError, te:
        # If this is a file-like object, try to
        # fstat its file descriptor
        try:
            thelen = str(os.fstat(body.fileno()).st_size)
        except (AttributeError, OSError):
            # Don't send a length if this failed
            if self.debuglevel > 0: print "Cannot stat!!"

    if thelen is not None:
        self.putheader('Content-Length', thelen)

    ...

def _send_output(self, message_body=None):
    ...
    if isinstance(message_body, str):
        msg += message_body
        message_body = None
    self.send(msg)
    if message_body is not None:
        #message_body was not a string (i.e. it is a file) and
        #we must run the risk of Nagle
        self.send(message_body)

    ...

def send(self, data):
    ...
    blocksize = 8192
    if hasattr(data,'read') and not isinstance(data, array):
        if self.debuglevel > 0: print "sendIng a read()able"
        datablock = data.read(blocksize)
        while datablock:
            self.sock.sendall(datablock)
            datablock = data.read(blocksize)
    else:
        self.sock.sendall(data)

My script.py:

#!/usr/bin/env python

from io import BytesIO

data = BytesIO()

print type(data.getvalue()

$./script.py

<type 'str'>

My solution:

#!/usr/bin/python

from urllib3 import HTTPConnectionPool, filepost
from io import BytesIO, SEEK_SET

pool = HTTPConnectionPool('web')

archive_name = '/var/archives/mock.tar.gz'

fields = {'uploadedfile' : (archive_name, open(archive_name).read())}

value, content_type = filepost.encode_multipart_formdata(fields)

headers = {'Content-Type' : content_type, 'Content-Length' : len(value)}

# or any temporary file. without Content-Length

data = BytesIO()

data.write(value)

data.seek(0, SEEK_SET)

resp = pool.urlopen('POST', '/upload/uploader.php', data, headers)

print resp.status

$./upload.py

200

/var/www/upload# ls

mock.tar.gz upload.html upload.php

gaal.dev
  • 61
  • 2
  • 3
2

Try encoding your message_body string:

msg += message_body.encode('utf-8')

This post really helped me when it comes to python strings and unicode.

Community
  • 1
  • 1
garnertb
  • 9,454
  • 36
  • 38
  • This error is probably being caused by the line "Q. Why am I am getting "The authenticated user has not installed the app with client id†error?" in your readme. A cheap solution to fixing this would be to replace "idâ€". – garnertb Apr 29 '12 at 14:31