Django - User uploaded S3 files in the view

Question

I have a page where users can upload PDF / image files to their profile. The model for these files is relativly straightforward:

class ResumeItemFile(models.Model):
    item = models.ForeignKey(ResumeItem, related_name='attachment_files')
    file = models.FileField(
    max_length=255, upload_to=RandomizedFilePath('resume_attachments'),
    verbose_name=_('Attachment'))
    name = models.CharField(max_length=255, verbose_name=_('Naam'), blank=True)

I am creating a view where all files linked to a profile (item) are gathered in a .zip file. I've got this working locally, but in production I run in the following error NotImplementedError: This backend doesn't support absolute paths.

The main difference is that on production the mediafiles are served through S3

MEDIA_URL = 'https://******.s3.amazonaws.com/'
STATIC_URL = MEDIA_URL

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

In my view I created a list of the ResumeItemFile in the attachments variable, which is a list of dicts that look like this: {'filename', ResumeItemFileObject}

            for file in attachments:

                storage = DefaultStorage()
                filename = file[1]
                file_extension = str(file[0].file).split('.')[-1]
                file_object = storage.open(file[0].file.path, mode='rb')                   
                filename, file_object.read())
                file_object.close()

Though this works fine locally, on staging it crashes on the file_object = storage.open(file[0].file.path, mode='rb') line.

If the backend does not support absolute paths, how I am to select the correct file? Does anyone have an idea of what I am doing wrong?

Thanks for your response! I've updated my initial comment with the required information, but the STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage' — Jasper, Feb 14 '18 at 10:57

score 8 · Accepted Answer · answered Feb 14 '18 at 11:07

I think that problem comes because in the s3boto storage class, the path() method is not implemented. As per the Django documentation,

For storage systems that aren’t accessible from the local filesystem, this will raise NotImplementedError instead.

Instead of file.path use file.name in your code.

# file_object = storage.open(file[0].file.path, mode='rb')    
file_object = storage.open(file[0].file.name, mode='rb')

score 1 · Answer 2 · answered Feb 14 '18 at 11:30

1

You may want to look into the File object. It allows you to manipulate files in a largely Pythonic manner, but leverages the Django project's storage settings. In my case, this allows me to use local, on-disk storage locally and S3 in production:

https://docs.djangoproject.com/en/2.0/ref/files/file/

This will abstract away a lot of the boilerplate you're writing. There is an example here:

https://docs.djangoproject.com/en/2.0/topics/files/#the-file-object

Good luck!

answered Feb 14 '18 at 11:30

FlipperPA

13,607
4
39
71

As an opinion based aside, I've had better luck with Django S3 Storage, if you're only using the S3 backend: https://github.com/etianen/django-s3-storage It is a simplified version of `django-storages` which only support one backend, S3. – FlipperPA Feb 14 '18 at 11:32

Django - User uploaded S3 files in the view

2 Answers2

Linked