boto connection works locally and in heroku python shell but fails in webapp

Question

I'm not sure how to diagnose this further. Heroku's evns are configured properly. Everything worked on my local machine line a charm. In production i get a internal server error when i try to use this functionality.

I booked up a live shell and everything worked fine there to:

In [5]: s3conn = boto.connect_s3(app.config["AWS_ACCESS_KEY_ID"],app.config["AWS_SECRET_ACCESS_KEY"])

In [6]: s3conn
Out[6]: S3Connection:s3.amazonaws.com

In [7]: bucket = s3conn.get_bucket(app.config["S3_BUCKET"])

In [8]: bucket
Out[8]: <Bucket: petalbucket>

In [10]: key = bucket.get_key('AGRAPH')

In [11]: key

In [12]: type(key)
Out[12]: NoneType



In [14]: key = bucket.get_key('charts/AGRAPH')

In [15]: key
Out[15]: <Key: petalbucket,charts/AGRAPH>

In [16]: seconds = 60*5

In [17]: url = key.generate_url(expires_in=seconds)

In [18]: url
Out[18]: 'https://petalbucket.s3.amazonaws.com/charts/AGRAPH?Signature=VL9Q9r0Q%2F5UbZDhW%2F%2BeUYFezSGU%3D&Expires=1357251512&AWSAccessKeyId=AKIAJBEQAKKMVPGYYNRA'

code:

 def download_s3(file_title):
        s3conn = boto.connect_s3(app.config["AWS_ACCESS_KEY_ID"],app.config["AWS_SECRET_ACCESS_KEY"])
        bucket = s3conn.get_bucket(app.config["S3_BUCKET"])
        key = bucket.get_key(file_title)
        seconds = 60*5
        url = key.generate_url(expires_in=seconds)
        return url

heroku logs:

2013-01-03T22:21:05+00:00 app[web.1]:     respiter = self.wsgi(environ, resp.start_response)
2013-01-03T22:21:05+00:00 app[web.1]:     response = self.make_response(self.handle_exception(e))
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/web_transaction.py", line 563, in __call__
2013-01-03T22:21:05+00:00 app[web.1]:     return wrapped(*args, **kwargs)
2013-01-03T22:21:05+00:00 app[web.1]: Traceback (most recent call last):
2013-01-03T22:21:05+00:00 app[web.1]:     result = application(environ, _start_response)
2013-01-03T22:21:05+00:00 app[web.1]: 2013-01-03 22:21:05 [7] [ERROR] Error handling request
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 105, in handle_request
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/object_wrapper.py", line 220, in __call__
2013-01-03T22:21:05+00:00 app[web.1]:     self._nr_instance, args, kwargs)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/web_transaction.py", line 485, in __call__
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1701, in __call__
2013-01-03T22:21:05+00:00 app[web.1]:     return self._nr_next_object(environ, start_response)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1689, in wsgi_app
2013-01-03T22:21:05+00:00 app[web.1]:     response = self.full_dispatch_request()
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request
2013-01-03T22:21:05+00:00 app[web.1]:     rv = self.handle_user_exception(e)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request
2013-01-03T22:21:05+00:00 app[web.1]:     rv = self.dispatch_request()
2013-01-03T22:21:05+00:00 app[web.1]:     self._nr_instance, args, kwargs)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/object_wrapper.py", line 220, in __call__
2013-01-03T22:21:05+00:00 app[web.1]:     self._nr_instance, args, kwargs)
2013-01-03T22:21:05+00:00 app[web.1]:     return wrapped(*args, **kwargs)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask_login.py", line 496, in decorated_view
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/petalapp/views.py", line 135, in add_pci_form
2013-01-03T22:21:05+00:00 app[web.1]:     return fn(*args, **kwargs)
2013-01-03T22:21:05+00:00 app[web.1]: AttributeError: 'NoneType' object has no attribute 'generate_url'
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/petalapp/aws/tools.py", line 58, in download_s3
2013-01-03T22:21:05+00:00 app[web.1]:     url = key.generate_url(expires_in=seconds)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/function_trace.py", line 81, in literal_wrapper
2013-01-03T22:21:05+00:00 app[web.1]:     url =download_s3(in_file + title)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/pre_function.py", line 14, in wrapper
2013-01-03T22:21:05+00:00 app[web.1]:     return self.view_functions[rule.endpoint](**req.view_args)
2013-01-03T22:21:05+00:00 app[web.1]:     return self.wsgi_app(environ, start_response)
2013-01-03T22:21:05+00:00 app[web.1]:     return wrapped(*args, **kwargs)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/object_wrapper.py", line 220, in __call__
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/function_trace.py", line 81, in literal_wrapper
2013-01-03T22:21:05+00:00 app[web.1]:     return wrapped(*args, **kwargs)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/object_wrapper.py", line 220, in __call__
2013-01-03T22:21:05+00:00 app[web.1]:     self._nr_instance, args, kwargs)
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/newrelic-1.9.0.21/newrelic/api/transaction_name.py", line 51, in literal_wrapper
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app
2013-01-03T22:21:05+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1344, in dispatch_request

My hunch is i'm not understanding something about boto's interaction with heroku. Or haven't set something correctly but i can't see what that could be from this information.

Did you check your requirements.txt for boto as a dependency? — Victor 'Chris' Cabral, Jan 03 '13 at 22:54

score 1 · Accepted Answer · answered Jan 03 '13 at 23:05

1

The stack suggests bucket.get_key(file_title) returns None which is the exact same response you get above when you've typed the wrong file name (i.e. when you look for 'AGRAPH' instead of 'charts/AGRAPH')

What is file_title set to? is it correct? I'd suggest some debugging to get the value of that.

answered Jan 03 '13 at 23:05

dannyclark

76
2

I had a ENV variable slightly wrong and so the file_title was off. – Drew Verlee Jan 04 '13 at 03:00

boto connection works locally and in heroku python shell but fails in webapp

1 Answers1