2

would you please help to run collect static to amazon s3?

this is my config file:

05_collect_admin_static:
    command: "python manage.py collectstatic --noinput"
    leader_only: true

in setting file i am using django-storages==1.1.8 and boto==2.32.1

INSTALLED_APPS += ('storages',)
AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_KEY']
AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

S3_URL = 'http://%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = S3_URL

and this is my log:

/var/log/cfn-init.log

   execute_from_command_line(sys.argv)
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 173, in handle_noargs
    collected = self.collect()
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 112, in collect
    handler(path, prefixed_path, storage)
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 317, in copy_file
    self.storage.save(prefixed_path, source_file)
  File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/files/storage.py", line 49, in save
    name = self._save(name, content)
  File "/opt/python/run/venv/lib/python2.7/site-packages/storages/backends/s3boto.py", line 392, in _save
    self._save_content(key, content, headers=headers)
  File "/opt/python/run/venv/lib/python2.7/site-packages/storages/backends/s3boto.py", line 403, in _save_content
    rewind=True, **kwargs)
  File "/opt/python/run/venv/lib/python2.7/site-packages/boto/s3/key.py", line 1286, in set_contents_from_file
    chunked_transfer=chunked_transfer, size=size)
  File "/opt/python/run/venv/lib/python2.7/site-packages/boto/s3/key.py", line 746, in send_file
    chunked_transfer=chunked_transfer, size=size)
  File "/opt/python/run/venv/lib/python2.7/site-packages/boto/s3/key.py", line 944, in _send_file_internal
    query_args=query_args
  File "/opt/python/run/venv/lib/python2.7/site-packages/boto/s3/connection.py", line 664, in make_request
    retry_handler=retry_handler
  File "/opt/python/run/venv/lib/python2.7/site-packages/boto/connection.py", line 1053, in make_request
    retry_handler=retry_handler)
  File "/opt/python/run/venv/lib/python2.7/site-packages/boto/connection.py", line 923, in _mexe
    request.body, request.headers)
  File "/opt/python/run/venv/lib/python2.7/site-packages/boto/s3/key.py", line 840, in sender
    http_conn.send(chunk)
  File "/usr/lib64/python2.7/httplib.py", line 805, in send
    self.sock.sendall(data)
  File "/usr/lib64/python2.7/ssl.py", line 229, in sendall
    v = self.send(data[count:])
  File "/usr/lib64/python2.7/ssl.py", line 198, in send
    v = self._sslobj.write(data)
socket.error: [Errno 104] Connection reset by peer

thanks. is it related to permission issue? i am using aws elastic beanstalk. thanks again.

user2673206
  • 215
  • 2
  • 12
  • FYI I read quite a lot of tutorial already but still cant figure it out whats going wrong. thanks a lot for your help. http://www.grigory.ca/2012/09/getting-started-with-django-on-aws-elastic-beanstalk/ http://blog.doismellburning.co.uk/2012/07/14/using-amazon-s3-to-host-your-django-static-files/ http://dontpanic.ghost.io/django-s3-configuration/ – user2673206 Aug 14 '14 at 05:38
  • Possible duplicate of [Connection reset by peer when using s3, boto, django-storage for static files](http://stackoverflow.com/questions/20436705/connection-reset-by-peer-when-using-s3-boto-django-storage-for-static-files) – megapctr Aug 18 '16 at 13:49

2 Answers2

2

Here's what fixed it for me (settings.py):

AWS_S3_HOST = 's3-us-west-2.amazonaws.com'

Normally, boto would use the default region. If your bucket isn't in the default region, AWS will respond with a redirect (and boto does not follow redirects).

Without this setting, smaller files got uploaded anyway - maybe the redirect is emitted only for the bigger files?

megapctr
  • 931
  • 1
  • 7
  • 19
1

it seems i finally found a solution.

this is the setting file:

#

AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')

STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

STATIC_URL = 'http://' + AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/'
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'

#

the mistake i made was:

wrong:
  - option_name: AWS_STORAGE_BUCKET_NAME
    value: bucket_name.s3-website-eu-west-1.amazonaws.com

correct:
  - option_name: AWS_STORAGE_BUCKET_NAME
    value: bucket_name

stupid me. took me two days to find this issue....

user2673206
  • 215
  • 2
  • 12