How can I do a replace with gridfs-stream?

Question

I'm using this code to do a file update:

app.post("/UploadFile", function(request, response)
{
    var file = request.files.UploadedFile;

    var name = request.param("Name");
    var componentId = request.param("ComponentId");

    console.log("Uploading: " + name);

    var parameters =
    {
        filename: name,
        metadata:
        {
            Type: "Screenshot",
            ComponentId: componentId
        }
    };

    grid.files.findOne( { "metadata.ComponentId" : componentId }, function(error, existing)
    {
        console.log("done finding");
        if (error)
        {
            common.HandleError(error);
        }
        else
        {
            if (existing)
            {
                console.log("Exists: " + existing._id);
                grid.remove({ _id: existing._id }, function(removeError)
                {
                    if (removeError)
                    {
                        common.HandleError(removeError, response);
                    }
                    else
                    {
                        SaveFile(file, parameters, response);
                    }
                });
            }
            else
            {
                console.log("new");
                SaveFile(file, parameters, response);
            }
        }
    });
});

function SaveFile(file, parameters, response)
{
    console.log("Saving");
    var stream = grid.createWriteStream(parameters);

    fs.createReadStream(file.path).pipe(stream);
}

Basically I'm checking for a file that has an ID stored in metadata. If it exists, I delete it before my save, and if not I just do the save. It seems to work only sporadically. I sometimes see two erroneous behaviors:

The file will be deleted, but not recreated.

The file will appear to be updated, but it won't actually be replaced until I call my code again. So basically I need to do two file uploads for it to register the replace.

It's very sketchy, and I can't really determine a pattern for if it's going to work or not.

So I'm assuming I'm doing something wrong. What's the right way to replace a file using gridfs-stream?

score 0 · Answer 1 · answered Dec 09 '13 at 01:02

It's difficult to say for sure from just the code you've provided (i.e. you don't show how the response to the app.post is ultimately handled), but I see several red flags to check:

Your SaveFile function above will return immediately after setting up the pipe between your file and the gridFS store. That is to say, the caller of the code you provide above will likely get control back well before the file has been completely copied to the MongoDB instance if you are moving around large files, and/or if your MongoDB store is over a relatively slow link (e.g. the Internet).

In these cases it is very likely that any immediate check by the caller will occur while your pipe is still running, and therefore before the gridFS store contains the correct copy of the file.

The other issue is you don't do any error checking or handling of the events that may be generated by the streams you've created.

The fix probably involves creating appropriate event handlers on your pipe, along the lines of:

function SaveFile(file, parameters, response)
{
    console.log("Saving");
    var stream = grid.createWriteStream(parameters);

    pipe = fs.createReadStream(file.path).pipe(stream);

    pipe.on('error', function (err) {
         console.error('The write of " + file.path + " to gridFS FAILED: ' + err);
         // Handle the response to the caller, notifying of the failure 
    });

    pipe.on('finish', function () {
         console.log('The write of " + file.path + " to gridFS is complete.');
         // Handle the response to the caller, notifying of success 
    });
}

The function handling the 'finish' event will not be called until the transfer is complete, so that is the appropriate place to respond to the app.post request. If nothing else, you should get useful information from the error event to help in diagnosing this further.

How can I do a replace with gridfs-stream?

1 Answers1