Django migrations and FileSystemStorage depending on settings

Question

In my Django app I use a FileSystemStorage for generated files. I initialize it like this:

import os
from urlparse import urljoin

from django.conf import settings
from django.core.files.storage import FileSystemStorage

gen_files_storage = FileSystemStorage(location=os.path.join(settings.MEDIA_ROOT, 'generated/'), base_url=urljoin(settings.MEDIA_URL, 'generated/'))

When I want to create a new file I use:

from django.core.files.base import ContentFile
from django.db import models

def next_number():
    # num = ...
    return num

gen_file = models.FileField(storage=gen_files_storage)
gen_file.save('file%s.txt' % next_number(), ContentFile(''))

That works fine. The only problem is that the FileSystemStorage's path is "hardcoded" in the Django migration. Because I use different settings for development (which changes) and production, often the manage.py makemigrations command generates a migration only because the path changed, although everything stays the same in the database.

I know there is a solution using a subclass of FileSystemStorage (see my answer below), but is there a better solution?

score 11 · Accepted Answer · edited Sep 30 '16 at 11:58

There is a solution involving a custom @deconstructible subclass of FileSystemStorage:

import os
from urlparse import urljoin

from django.conf import settings
from django.core.files.storage import FileSystemStorage
from django.utils.deconstruct import deconstructible

@deconstructible
class MyFileSystemStorage(FileSystemStorage):
    def __init__(self, subdir):
        self.subdir = subdir
        super(MyFileSystemStorage, self).__init__(location=os.path.join(settings.MEDIA_ROOT, self.subdir), base_url=urljoin(settings.MEDIA_URL, self.subdir))

    def __eq__(self, other):
        return self.subdir == other.subdir

Then I can initialize the storage like this:

import os
from urlparse import urljoin

from django.conf import settings
from django.core.files.storage import FileSystemStorage

gen_files_storage = MyFileSystemStorage('generated/')

This way Django migrations won't notice changes in my settings. Is there a better way though?

I suppose it's like this: https://docs.djangoproject.com/en/1.9/howto/custom-file-storage/, https://code.djangoproject.com/ticket/22337 — Daviddd, Dec 18 '15 at 09:35
Here is another option: https://github.com/translate/pootle/commit/8ff2463f0b1f1771595334df9ff9f7ba4ec33ae5#diff-0073684248aab77bd945bf6209970a09R34 — jaywink, Jul 19 '16 at 08:28

score 8 · Answer 2 · answered Jul 14 '21 at 17:58

Upgrading to Django 3.1+ fixes this: https://docs.djangoproject.com/en/3.2/releases/3.1/#file-storage

Just pass a callable into the storage argument.

from django.db import models
from django.conf import settings
from django.core.files.storage import get_storage_class


def _get_storage():
    storage_class = get_storage_class(settings.MY_STORAGE_CLASS)  # ie. 'django.core.files.storage.FileSystemStorage'
    return storage_class()

class MyModel(models.Model):
    myfile = models.FileField(max_length=255, blank=True, storage=_get_storage)

This answer is great! In most cases I think people won't even need the get_storage_class functionality if they only plan on using FileSystemStorage and changing the location. The simplest use case is just constructing the FileSystemStorage with a variable location within the function _get_storage. — Dre, May 09 '23 at 17:41

score 2 · Answer 3 · answered Sep 12 '18 at 22:01

The solution is to never run makemigrations on production. Run migrate all you want on production servers, but ignore warnings about running makemigrations if they pertain to this issue.

Think about it: makemigrations generates Python code, so running it on production would be the same as developing on that server. Depending on your server setup, your production site will likely serve those files correctly regardless of the makemigrations warning.

score 2 · Answer 4 · answered Dec 04 '18 at 21:32

My problem was related, but slightly different. The storage class used by the field can change based on settings: the default locally, remote storage in production. I implemented a subclass of FileField that ignores the storage kwarg when deconstructing the field for migration generation.

from django.db.models import FileField

class VariableStorageFileField(FileField):
    """
    Disregard the storage kwarg when creating migrations for this field
    """

    def deconstruct(self):
        name, path, args, kwargs = super(VariableStorageFileField, self).deconstruct()
        kwargs.pop('storage', None)
        return name, path, args, kwargs

It can be used like this:

class MyModel(models.Model):
    storage = get_storage_class(getattr(settings, 'LARGE_FILE_STORAGE', None))()

    file = VariableStorageFileField(blank=True, null=True, storage=storage)

Django migrations and FileSystemStorage depending on settings

4 Answers4

Linked