14

I followed every QA suggestions found on SO and in different blogs, Everything works ok on my dev machine and nothing works on heroku.

here are my settings:

DEFAULT_FILE_STORAGE = 'arena.utils.MediaRootS3BotoStorage' # media files
# storage

AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')
AWS_PRELOAD_METADATA = True # necessary to fix manage.py collectstatic command to only upload changed files instead of all files

S3_URL = 'https://%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
MEDIA_URL = S3_URL + '/media/'

STATIC_URL = S3_URL + '/static/'

ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'


COMPRESS_URL = STATIC_URL
COMPRESS_OFFLINE = True
COMPRESS_STORAGE = 'utils.CachedS3BotoStorage'
STATICFILES_STORAGE = COMPRESS_STORAGE

When i run collectstatic/compress everything is ok, i see the files being collected to S3 and put in proper places. I see the manifest file.

Loading any page with compressor tag, show an error OfflineGenerationError: You have offline compression enabled but key "d2a53169c44dec41ce3ee7da19b2b9d4" is missing from offline manifest. Running python manage.py compress again solves nothing. when i check the manifest file, indeed the key it looks for doesn't exist.

What is going wrong here?

Question i already checked:

How to configure django-compressor and django-staticfiles with Amazon's S3?

Django Compressor with S3 URL Heroku

Configuring django-compressor with remote storage (django-storage - amazon s3)

Community
  • 1
  • 1
Neara
  • 3,693
  • 7
  • 29
  • 40
  • I think this has something to do with how you add less files to your template. I added them dynamically in development, which was fine. When you use the `manage.py compress` command, it doesn't see the dynamically added ones, and cannot compress those. – Jacob Valenta Jan 22 '14 at 06:01
  • @JacobValenta i dont have less files in templates. i have a compiler working automatically on every save in Pycharm. `manage.py compress` works on heroku, doesn't throw any errors, but when i try to load a page i get same error every time. As if it cant read the manifest or something – Neara Jan 22 '14 at 07:45

3 Answers3

1

On my side I have very similar config, and I'm successfully using compressor for more than 2 years.

settings.py

COMPRESS_STORAGE = 'MyAwesomeApp.app.CachedS3BotoStorage.CachedS3BotoStorage'

AWS_ACCESS_KEY_ID = '#######'
AWS_SECRET_ACCESS_KEY = '########################+#########+BqoQ'
AWS_STORAGE_BUCKET_NAME = 'myAmazonS3cdn.myawesomewebsite.com'
AWS_S3_SECURE_URLS = False
AWS_QUERYSTRING_AUTH = False

COMPRESS_ROOT = 'MyAwesomeApp/static'
STATIC_ROOT = 'MyAwesomeApp/static/javascript'
COMPRESS_OUTPUT_DIR = 'compressed'
STATICFILES_STORAGE = COMPRESS_STORAGE

STATIC_URL = "http://myAmazonS3cdn.myawesomewebsite.com/"
COMPRESS_URL = STATIC_URL
COMPRESS_ENABLED = True

CachedS3BotoStorage.py

from django.core.files.storage import get_storage_class
from storages.backends.s3boto import S3BotoStorage

from django.core.files.base import File

class CachedS3BotoStorage(S3BotoStorage):
    """
    S3 storage backend that saves the files locally, too.
    """
    def __init__(self, *args, **kwargs):
        super(CachedS3BotoStorage, self).__init__(*args, **kwargs)
        self.local_storage = get_storage_class("compressor.storage.CompressorFileStorage")()

    def save(self, name, content):
        name = super(CachedS3BotoStorage, self).save(name, content)
        self.local_storage._save(name, content)
        return name

I'm running python managep.py compress locally, and having manifest generated on my static files directory. Heroku only deals with the collecstatic and delivers the most recent manifest version to my cdn.

Regards,

André Teixeira
  • 2,392
  • 4
  • 28
  • 41
  • I had an issue, and adding `AWS_QUERYSTRING_AUTH = False` is finally making it work. Since the signature changes over time otherwise. – KVISH May 20 '14 at 16:09
  • It is not well documented, but before using the AWS_QUERYSTRING_AUTH paramenter I was stucked on "Request has expired" error. – André Teixeira May 22 '14 at 20:52
  • Yes, what I did previously was `AWS_QUERYSTRING_EXPIRE = 63115200`. That sets the request expiration to 2 years from now. But even that's not the best solution. – KVISH May 22 '14 at 20:54
1

I completed the above solution with some lines, to fix the problem that create many (multiples) manifest_%.json in Amazon S3

in setting.py:

STATICFILES_STORAGE = 'your_package.s3utils.CachedS3BotoStorage'

in s3utils.py:

from storages.backends.s3boto import S3BotoStorage
from django.core.files.storage import get_storage_class

class CachedS3BotoStorage(S3BotoStorage):
    """
    S3 storage backend that saves the files locally, too.
    """
    location = 'static'

    def __init__(self, *args, **kwargs):
        super(CachedS3BotoStorage, self).__init__(*args, **kwargs)
        self.local_storage = get_storage_class(
            "compressor.storage.CompressorFileStorage")()

    def url(self, name):
        """
        Fix problem images admin Django S3 images
        """
        url = super(CachedS3BotoStorage, self).url(name)
        if name.endswith('/') and not url.endswith('/'):
            url += '/'
        return url

    def save(self, name, content):
        name = super(CachedS3BotoStorage, self).save(name, content)
        self.local_storage._save(name, content)
        return name

    # HERE is secret to dont generating multiple manifest.json and to delete manifest.json in Amazon S3
    def get_available_name(self, name):
        if self.exists(name):
            self.delete(name)
        return name
Mateus Padua
  • 183
  • 2
  • 8
0

I found a git repository that contains post_compile hooks to solve this problem. It runs compress after Heroku built the Django app (and also installs lessc if you need less in your compressor settings).

https://github.com/nigma/heroku-django-cookbook

yellowcap
  • 3,985
  • 38
  • 51