0

What I need to achieve is to allow a user to upload files to Google Cloud Storage from his browser: the user select a file, a FormData containing the file is created and sended to a Cloud Function. The Cloud Function then sends the data to Google Cloud Storage. This piece of code works well for small files. But when I try to upload bigger files (30mo is the size I usually want to send) I have this error

PayloadTooLargeError: request entity too large

It looks like cloud function check the size of the uploaded data even if try to make the limit bigger. Is there any way to go beyond the 10mo limit ?

app.use(bodyParser.json({limit: '50mb'}))
app.use(bodyParser.urlencoded({limit: '50mb', extended: true }));

app.post("/upload-my-file", async (req, res) => {
    try {
        await authentChecker(req.headers)
        const busboy = new Busboy({headers: req.headers});

        busboy.on('file', (fieldname, file, filename) => {
            const storage = new Storage();
            const myBucket = storage.bucket('my-bucket-name');
            const newBucketFile = myBucket.file(filename);

            //I don't write a temp file on disk, I directly upload it
            file.pipe(newBucketFile.createWriteStream())
            .on('error', function(err) {
                console.log(err)
            })
            .on('finish', function() {
                console.log("finish")
            });

        })
        // Triggered once all uploaded files are processed by Busboy.
        // We still need to wait for the disk writes (saves) to complete.
        busboy.on('finish', async () => {
            res.send();
        });
        busboy.end(req.rawBody);
    } catch (error) {
        next(error)
    }
})
Thomas Fournet
  • 660
  • 1
  • 6
  • 23

1 Answers1

0

As per the github you need to add "extended: true" for bodyParser.json. Please see body-parser module documentation for more about this.

bodyParser = { json: {limit: '50mb', extended: true}, urlencoded: {limit: '50mb', extended: true} };

OR

app.use(bodyParser.json({limit: '10mb', extended: true}))

app.use(bodyParser.urlencoded({limit: '10mb', extended: true}))

Mahboob
  • 1,877
  • 5
  • 19
  • Thanks for your answer but this solution doesn't work, I still have the same "PayloadTooLargeError" error. I tried what people suggested on this github page. It looks like they may not be using Cloud Function which should explain why they don't have those limitations – Thomas Fournet Jan 21 '21 at 08:59