-1

I am running a server in django and I want to serve a file. I am storing the file under '/upload/directory/filename' and I return it using

from django.shortcuts import redirect

file_path = '/upload/directory/filename'
return redirect(file_path)

However, the file appears to have been cached to the first version that had been placed locally and is never updated. Even if I remove the file, the file is still served. I checked that if I change the path to 'upload/directory_2/filename then I correctly get my new file. What is going on and how can I fight this ?

This is happening locally and I am making a direct server request (hence there is no possibility of browser caching or anything else).

Additional information:

I understand that maybe I should be using static files, although for instance this answer suggests that it is quite debatable for files that I am uploading myself.

When I say "I want to serve files with django" I just mean that I have associated a file path to a particular entity in my database (using models.FileField) and based on what the user requests I want to return this file. I kind of doubt this is a clear cut for using static files in that case.

There are many workarounds to my issue, like generating unique filenames every time I want to "clear my cache" or explicitly opening the file:

with open(absolute_file_path) as file:
    response = HttpResponse(file.read(), content_type='application/octet-stream')

My question was about understanding why the particular piece of code above does what it does, i.e. leads to data caching, and how to prevent this.

vib
  • 2,254
  • 2
  • 18
  • 36
  • 2
    Why are you serving files with django? – Andrey Shipilov May 30 '17 at 09:26
  • I have a server running in django and I use it to associate a SQL database with files. I have an ios application that needs to connect to this server, giving as argument the name of an object in the database, and the django server should redirect the appropriate file. – vib May 30 '17 at 09:29
  • Meanwhile, I found a solution: instead of `redirect`, do `with open(absolute_file_path) as file: response = HttpResponse(file.read(), content_type='application/octet-stream')` but I'd still be interested in why redirect is not working as expected (or is it?) and how I could clean this cache. – vib May 30 '17 at 09:31
  • 3
    I'm just warning you, this is extremely wrong. – Andrey Shipilov May 30 '17 at 09:31
  • 1
    Why ? Can you elaborate a bit more? – vib May 30 '17 at 09:32
  • 1
    https://docs.djangoproject.com/en/1.11/howto/static-files/#configuring-static-files first green block. – Andrey Shipilov May 30 '17 at 09:32
  • On the other hand you can read for instance first answer here: https://stackoverflow.com/questions/5016589/what-is-the-difference-between-static-files-and-media-files-in-django my use case seems to fall into the media files case, and so I am not sure whether moving to the static solution is the best thing to do. In which case I guess my question remains open... – vib May 30 '17 at 19:38
  • Just to add to why this might be "extremely wrong" -- it depends on how you're creating `absolute_file_path`, but you have to be careful here, because if you're just passing user input straight to `os.path.join(static_root, rel_path)` they'll be able to use specify an absolute path or a path with `../../` in it to access sensitive files higher in the directory tree. – Ben Hoyt Jun 02 '17 at 17:52
  • I am just building a server that is responding to authentified request from an ios app so I should have full control over the arguments passed and I am hoping that I don't have these kind of issues. But if there's anything obvious I should be wary of I'd be very happy to know. As I see it there is not a fondamental difference between those files I upload and files that the user could upload and I am note sure they should really be treated as static file. So I would just like to know what is the efficient way to serve the files in this case while avoiding stale cached files. – vib Jun 02 '17 at 18:01
  • Can you add your relevant settings to the question? Did you run `collectstatic` at any point in time? – Igonato Jun 02 '17 at 18:07
  • Not sure what the relevant settings are? I had run `collectstatic` in the past but it is acting on a different subdirectory than where I am placing my file. – vib Jun 02 '17 at 18:08
  • Is the file in that directory? What happens if you remove it from there? – Igonato Jun 02 '17 at 18:09
  • It is still returned. – vib Jun 02 '17 at 18:15
  • Ok. Please show your `STATICFILES_DIRS`, `MEDIA_ROOT`, `STATIC_ROOT` settings. If you're using the included `django.contrib.staticfiles.views.serve` view to serve the files, then it doesn't do any caching at all. Your file must be served from somewhere – Igonato Jun 02 '17 at 18:18
  • Also, to clarify. "uploaded file" in your question means uploaded through Django or do mean you just uploading it to a remote folder? – Igonato Jun 02 '17 at 18:20
  • As I am saying, I am not serving these files as static file. They live in my `DATA_ROOT` (defined below) and are returned through redirect(their_path). STATIC_ROOT = os.path.abspath(os.path.join(BASE_DIR, 'static')) DATA_ROOT = os.path.join(BASE_DIR, 'my_files') MEDIA_ROOT = os.path.join(BASE_DIR, 'uploads'). Uploaded means I placed them in a local folder, without django. – vib Jun 02 '17 at 18:21
  • Can you edit the question instead? – Igonato Jun 02 '17 at 18:22
  • I don't think DATA_ROOT is a valid settings option. What version of Django are you using and what 3rd party libraries if any? – Igonato Jun 02 '17 at 18:26
  • Oh, I just realized. Do you understand ho `redirect` is working? It takes a *url* path. If you are trying to give it a path on your filesystem it's not going to work. – Igonato Jun 02 '17 at 18:30
  • Sorry, I think I confused things unecessarily. I am correctly using the url of my fle, as demonstrates the fact that if I change the name of the directory I get the correct file. Really, I think all the elements are in my question. – vib Jun 02 '17 at 19:00
  • Which view serves the uploaded files? Do you use django.views.static.serve? How do you upload the file? – emulbreh Jun 03 '17 at 18:10
  • You are using chrome? It's not server cache, just client side cache. Also your line : HttpResponse(file.read(), content_type='application/octet-stream') is not wrong. What the doc says however is that you should serve the files with something like Nginx and not django. I am not sure if that is your case right now. But to simply answer your question, client cache, your code is fine. – William R. Marchand Jun 03 '17 at 23:55
  • 5
    If this didn't have a bounty, it would already be closed. https://stackoverflow.com/help/mcve – e4c5 Jun 04 '17 at 12:56
  • I am not using Chrome, it is an API request, therefore there cannot be any client side cache. It seems like many people do not like that the problem doesn't fit the explanations that they would like to give but I can't change it... – vib Jun 05 '17 at 14:01
  • I don't know if you are deploying with nginx, but have a look at the `sendfile off;` option on nginx. In Any case the approach is not the best, you should use a model to wrap your file and not use django as static file server. – Karim N Gorjux Jun 08 '17 at 08:08

2 Answers2

-1

You must use django static facility to serve static files. On your local machine this is setup properly in settings to point a folder static in your app then you can point to the file using the static function.

All is explained here:

https://docs.djangoproject.com/en/1.11/howto/static-files/

In an url you access the code:

from django.templatetags.static import static
url_to_file = static('some_app/path/to_file')

On a production machine the static files are served by a web server or a specific service like aws S3 or similar but not from django! For this reason the static facility is a must.

To avoid the cache, in case you have this problem, have a look at the never_cache decorator: https://docs.djangoproject.com/en/1.11/topics/http/decorators/#caching

Karim N Gorjux
  • 2,880
  • 22
  • 29
-1

There is a special HttpResponse in Django that can help you to serve a file easily.

def file_dowload(request):
    file_name = request.GET.get('file_name', '')
    try:
        temp = open(file_name, 'rb')
    except IOError:
        temp = None
    if not temp:
        return HttpResponse("Not found")
    response = FileResponse(temp, content_type='text/plain')

    return response
Ezequiel Salas
  • 59
  • 5