3

Sorry if the question is simple, I am new to web development and self-hosted servers.

We have a self hosted website, which is supposed to have a button to download a large zip file (1 GB). For this, we have a simple solution in the index.html file:

<form action="path/to/file.zip" style="display: inline;">
    <button type="submit" class="btn btn-light">DOWNLOAD</button>
</form>

This usually worked fine. The problem is, whenever multiple visitors try to download the file at the same time, the server runs out of RAM and the website crashes. This is because every time someone clicks on the download button, the file appears to be loaded on RAM during the whole download process. When multiple visitors click the Download button, the file is loaded on RAM multiple times. As additional info, the website is built with python+Flask, and the server has 12 GB of RAM.

I have seen that in this answer (How to stop Apache from crashing my entire server?) they suggest to "take load off Apache for long-running processes", but I am not sure how to achieve that. Is there any solution you can suggest to solve this problem?

Thank you very much in advance.

djdomi
  • 1,599
  • 3
  • 12
  • 19
eugenio_chisari
  • 41
  • 3
  • 2
    Consider hosting the file somewhere off-server, like Amazon S3. – ceejayoz Jul 07 '21 at 15:04
  • 1
    That doesn't sound right at all - what actual webserver engine are you using? – Chopper3 Jul 07 '21 at 15:11
  • 1
    even file file would have 50gb ore more it should stream it instead of opening – djdomi Jul 07 '21 at 15:13
  • 2
    @Chopper3 I'd bet the requests are going through the app's front controller rather than direct to the webserver. – ceejayoz Jul 07 '21 at 15:20
  • 1
    @Chopper3 we use Nginx. @ djdomi could you point me to some resource explaining how to do that? Thank you all. – eugenio_chisari Jul 07 '21 at 15:31
  • @eugenio_chisati if you using plane nginx, post the nginx conf, this is not a normal behavior for nginx, you could try to use `directio 50m` to verify if your still running ooom – djdomi Jul 07 '21 at 19:32
  • 1
    Hello everyone, thank you very much for your feedback, it was very helpful for me to be able to ask around the right questions and find an answer. It turns out, our nginx server was simply forwarding the requests from the public address to the local address in our server; the app was then actually using a Tornado web server. So the solution was simply to use nginx directly to serve the static files, and only forwarding to the tornado server for the non-static part. This solved the memory problem. Thank you again for the help and for putting up with my beginner questions :) – eugenio_chisari Jul 14 '21 at 17:25

1 Answers1

1

thank you very much for your feedback, it was very helpful for me to be able to ask around the right questions and find an answer.

It turns out, our nginx server was simply forwarding the requests from the public address to the local address in our server; the app was then actually using a Tornado web server. So the solution was simply to use nginx directly to serve the static files, and only forwarding to the tornado server for the non-static part. This solved the memory problem.

Thank you again for the help and for putting up with my beginner questions :)

eugenio_chisari
  • 41
  • 3