3

i'm building a restful web service with Django as the backend and am currently concerned with scaling. I want to have an architecure like this: http://www.djangobook.com/en/2.0/_images/scaling-4.png where i can just start up more django instances on high load. My Problem is that i don't know how to externalize the user uploaded content (in django refered to as Media).

My Basic ideas are: Amazon S3, FTP Upload, SMT or NFS Mount, or some kind of file db like GridFS.

Because the EU canceled the Save Harbour Agreement and there is a lot of legal uncertainty for EU companies to put user data on non EU servers, i cant use Amazons S3 or any non EU based storage solution. I have looked at Minio, a S3 API compatible storage server. I could only get the posts to minio working with the latest (non stable) release, but not the gets. Other S3 compatible servers like CEPH are "too big" for my usecase.

I thought of uploading the files to an ftp server and serve them from there with nginx. The Problem is, that django-storages ftp functionality does not appear to be stable and django-resto is deprecated and no longer maintained.

SMB/NFS mounts are unstable and django would loose data it whats to write, when the mount is unavailabe due to network failures.

GridFS i could not get to work because it uses mongodb, which is non relational and django is relational. The non-relational django fork is not well maintained. In terms of serving files from GridFS there is nginx-gridfs and mod_gridfs, both are not maintained anymore.

Do you have any ideas on how to externalize django's media?

Jonas Kreusch
  • 316
  • 1
  • 15
  • I'm not sure of the relevance of mongodb being non-relational. Using gridfs for your static files doesn't require you to use mongodb for the rest of your data. – Daniel Roseman Mar 14 '16 at 16:00
  • Okay, good point. But still i would not be able to serve the media from gridfs because both nginx-gridfs and mod_gridfs are outdated – Jonas Kreusch Mar 14 '16 at 16:15
  • Any progress with this problem? Currently I'am trying to use Minio with my django project. – belek Jun 28 '16 at 09:16

1 Answers1

1

Well, I solved my problem by using Minio library. Created special storage class, which inherits django's FileStorageClass. Overrided url() and save() methods. Moved it to external module.

belek
  • 907
  • 9
  • 35