Can we limit upload file size without uploading it fully?

Question

I'm using multer v1.3.0 with express v4.15.4.

I have used fileSize limits like below

multer({
    storage: storage,
    limits: {fileSize: 1*1000*50, files:1},
    fileFilter: function (req, file, cb) {
        if (_.indexOf(['image/jpeg', 'image/png'], file.mimetype) == -1)
            cb(null, false);
        else
            cb(null, true)
    }
});

In this case I dont think limits are working. Cause not getting any LIMIT_FILE_SIZE error.
If I remove fileFilter then I'm getting LIMIT_FILE_SIZE error.
But in both cases first the whole file getting uploaded & then fileSize is checked.

Its not good at all to upload a 1GB of file and then check its of 1MB or not. So I want to know if there has any way to stop the uploading in the middle when file size limit exceeds. I don't want to rely on Content-Length.

try to change the `fileSize` to 1 and try again, let me know — Ahmed Can Unbay, Sep 10 '17 at 13:10
@turmuka same. I have tested with a 33MB file, after 13581.00 ms it says Error: File too large — Jayadratha Mondal, Sep 10 '17 at 13:18
try to make the max files size `250000` I have made a wrong calculation, as stated in the answer down below, it should quit receiving streams when it reaches the file size — Ahmed Can Unbay, Sep 10 '17 at 13:31

score 0 · Answer 1 · answered Sep 10 '17 at 13:24

0

From looking through the multer and busboy source it looks like it calculates the data size as a running total and stops reading the stream as soon as the fileSize is reached. See the code here:

https://github.com/mscdex/busboy/blob/8f6752507491c0c9b01198fca626a9fe6f578092/lib/types/multipart.js#L216

So while 1GB might be uploaded it shouldn't be saved anywhere.

If you want to stop the upload from continuing I think you'd have to close the socket though I haven't experimented with this myself. I'm not aware of any way to respond cleanly while the request is still trying to upload data.

answered Sep 10 '17 at 13:24

skirtle

27,868
4
42
57

yes the 1GB file will not be saved but the buffer will eat memory. Yes I'm trying to close the socket – Jayadratha Mondal Sep 10 '17 at 13:52
There was no measurable change to memory consumption during my test uploads (I used two separate physical machines for testing, not `localhost`). After a bit more digging it seems that while `busboy` emits its `limit` event straight away the handler in `multer` won't trigger an error until the file is finished uploading, so hooking in code to close the connection is difficult. It should be possible using `busboy` directly. Going through a webserver like nginx might offer a better way to protect against malicious users but if someone wants to fire dodgy requests at you it's hard to stop them. – skirtle Sep 10 '17 at 20:09
so you are advising to use a nginx or apache as proxy? Will not they upload the whole file if upload size exceeds limit? – Jayadratha Mondal Sep 10 '17 at 20:16

score 0 · Answer 2 · answered May 29 '21 at 06:14

You can use a javascript script to prevent users uploading all of the 1GB and getting a file size exceeded error. However, all client-side checks can be bypassed, so you should still enforce the file limit on the backend.

Your code is correct, it should work as intended. I am guessing you are worried about the file getting uploaded. There are no workarounds for that since Multer checks the size after upload.

Here is the javascript you can put in to prevent someone from uploading for your client-side code.

function ValidateSize(file) {
    const FileSize = file.files[0].size / 1024 / 1024;
    if (FileSize > 20) {
        alert('File size exceeds 20 MB');
        document.getElementById('formFile').value = null;
    }
}

<label for="formFile" class="form-label">Select file to upload (Max File Size: 20MB):</label>
<input class="form-control" type="file" onchange="ValidateSize(this)" id="formFile" name="file">

Can we limit upload file size without uploading it fully?

2 Answers2