1

I am writing an application that will be running on GCE and am using the google-api-go-client from https://code.google.com/p/google-api-go-client/ to store data in GCS.

I have a GAE application running as well that the GCE application communicates with.

I can run my GCE application locally and communicate with the live GCS servers and store files and retrieve them.

Now I am trying to locally store files in GCS so that I can run a test environment.

I see that devappserver supports simulating GCS via blobstore. However I can't seem to figure out how to point my GCE application at the local devappserver for GCS storage.

I am running a devappserver through 'goapp serve' with my GAE application.

I have tried setting the basepath and that results in 500 errors from the devappserver.

service, err := storage.New(httpClient)
service.BasePath = "http://localhost:8080/_ah/gcs/"

results in 

ERROR    2014-10-23 19:14:08,692 module.py:727] Request to '/_ah/gcs/b/default/o/6051711999279104' failed
Traceback (most recent call last):
  File "/home/ben/go_appengine/google/appengine/tools/devappserver2/module.py", line 723, in _handle_request
    return handler.handle(match, environ, wrapped_start_response)
  File "/home/ben/go_appengine/google/appengine/tools/devappserver2/wsgi_handler.py", line 59, in handle
    return self._wsgi_app(environ, start_response)
  File "/home/ben/go_appengine/google/appengine/tools/devappserver2/gcs_server.py", line 56, in __call__
    status_message = httplib.responses.get(e.args[1], '')
IndexError: tuple index out of range
INFO     2014-10-23 19:14:08,693 module.py:652] default: "GET /_ah/gcs/b/default/o/6051711999279104?alt=json HTTP/1.1" 500 -

I can point my browser to this URL

http://localhost:8080/_ah/gcs/bucket

and get an XML result:

<ListBucketResult xmlns="http://doc.s3.amazonaws.com/2006-03-01">
    <IsTruncated>False</IsTruncated>
</ListBucketResult>

I have seen several posts about using GCS locally but I can't seem to get it working correctly. Is there something I am doing wrong here? Or does devappserver not support GCS or maybe its just the json api it doesn't support?

Grokify
  • 15,092
  • 6
  • 60
  • 81
Ben Echols
  • 479
  • 2
  • 8

1 Answers1

2

I want to make sure we're clear on the distinction between the "Google APIs Client Library for Go", which is a general-purpose Google API client for Go that can be used to access the Google Cloud Storage JSON API, and App Engine's "Google Cloud Storage Go Client", which is a client library specific to App Engine users who want to use Google Cloud Storage.

There are two Google Cloud Storage APIs: JSON and XML. The general purpose Google APIs Client Library is compatible with the JSON API. The App Engine Google Cloud Storage Go Client speaks XML API.

The App Engine client libary fully supports local operations. This is accomplished by providing a local implementation of the Google Cloud Storage XML API, which the client library uses. Unfortunately, only the XML API exists locally, and thus only the App Engine client library works locally. Pointing the other client to it will fail because the XML API doesn't understand the incoming JSON API requests.

Your two best options are to do one of the following:

  1. Use the Google Cloud Storage Go Client Library for App Engine. Download instructions are here.
  2. Instead of using a local implemention, use the real Google Cloud Storage with a test bucket.
Brandon Yarbrough
  • 37,021
  • 23
  • 116
  • 145