NodeJS Buffer Base64 File Limits

Question

Tried to upload an image using Base64 but if the string is long the bottom of the stored image is blank white.

upload.js

var uuid = require('node-uuid');
var fs = require('fs');

exports.uploadImage = function(base64Str, callback) {
    var filename = uuid.v1()+'.png';
    var bitmap = new Buffer(base64Str, 'base64');
    fs.writeFileSync(filename, bitmap);
    callback(filename);
};

server.js

var fs = require('fs');
var restify = require('restify');  
var server = restify.createServer();
server.use(restify.bodyParser());

var upload = require('./modules/upload');

server.post('/images', function(req, res) {
    upload.uploadImage(req.params.myImage, function(filename) {
        console.log('processing image');
        res.send(filename);
        res.end();
    });
});

server.listen(3000);

It works for a string of length 499016 bytes but not if the string is 847508 bytes. Is there a documented size limit and, if not, how can I upload and un-encode longer strings?

if the image comes from a browser, i would suspect it more than node... — dandavis, May 01 '15 at 20:46
I am converting the image using an online encoder then passing it to node using Chrome Postman. I suspected the online encoder tool so used a second one but got the same result. — Mark Tyers, May 01 '15 at 20:48
make sure you are writing as binary, maybe some images fool the encoding detector, and you don't specify "binary"... — dandavis, May 01 '15 at 20:51
Small images decode perfectly and larger images decide partially yet remember the overall image size. Is there a timeout issue with NodeJS? — Mark Tyers, May 01 '15 at 20:53
there should not be, but you should use async anyway. perhaps the OS is reporting the write complete before the HDD actually archives it. does delaying the turn-around help? — dandavis, May 01 '15 at 20:55

Andrew Lavers · Accepted Answer · 2015-05-01T22:05:36.417

You might be hitting a request size limit. One alternative would be to upload your image as multipart/form-data. You can do this with the mapFiles options in restify.bodyParser, and you can test it by choosing file instead of text in your Postman form-data.

Edited: fs.rename might make more sense here for testing purposes. In a practical application you might want to save uploaded files elsewhere, on s3 perhaps.

var fs = require('fs');
var restify = require('restify');
var server = restify.createServer();
server.use(restify.bodyParser({
    mapFiles: true
}));

var uuid = require('node-uuid');
var fs = require('fs');

var uploadImage = function(path, callback) {
    var filename = uuid.v1() + '.png';
    fs.rename(path, filename, function(err) {
        if (err) {
            throw err;
        }
        callback(filename);
    });
};

server.post('/images', function(req, res) {
    uploadImage(req.files.myImage.path, function(filename) {
        res.send(filename);
        res.end();
    });
});

server.listen(3000);

Because my temp dir was on a different partition I got an EXDEV error. There were two possible changes to the above, either move my temp dir `http://stackoverflow.com/questions/21071303/node-js-exdev-rename-error` or substitute the `npm mv` module for `fs.rename`. I chose the latter and everything works well. — Mark Tyers, May 03 '15 at 08:00
Whilst the answer works well, I would still like to understand how to upload a large image base64 encoded. I have tried setting the bodyParser hoping this would fix things. Has anyone got any ideas? — Mark Tyers, May 04 '15 at 09:12

NodeJS Buffer Base64 File Limits

upload.js

server.js

1 Answers1