Not found error in bucket.createReadStream

Question

I just started using gcloud-node in an ExpressJS web app and have run in to what seems to be a very basic issue but I can't figure it out.

I can upload files no problem. I use:

fs.createReadStream(req.files.file.path)
    .pipe(bucket.createWriteStream(name))

Looking in my google developer console I see the file there in my bucket, for example with a name: 54314eddacd67fc858bf473f/543ba7bb387b56b908f2b1e4.jpg

When I try to download using that name i.e.:

bucket.createReadStream(file.name)
            .pipe(res)
             .on('error', function(err) {
                    errorResponseHandler.handleError(res, err);
                });

I get this error:

uncaughtException: Not Found Error: Not Found at Object.handleResp (/Users/ruairiobrien/Dev/testApp/node_modules/gcloud/lib/common/util.js:168:14) at Bucket.makeReq_ (/Users/ruairiobrien/Dev/testApp/node_modules/gcloud/lib/storage/index.js:580:10) at Request.Connection.req [as _callback] (/Users/ruairiobrien/Dev/testApp/node_modules/gcloud/lib/common/connection.js:251:16) at Request.init.self.callback (/Users/ruairiobrien/Dev/testApp/node_modules/gcloud/node_modules/request/request.js:199:22) at Request.emit (events.js:98:17) at Request.onResponse (/Users/ruairiobrien/Dev/testApp/node_modules/gcloud/node_modules/request/request.js:1160:14) at Request.emit (events.js:117:20) at IncomingMessage.Request.onResponse.strings (/Users/ruairiobrien/Dev/testApp/node_modules/gcloud/node_modules/request/request.js:1111:12) at IncomingMessage.emit (events.js:117:20) at _stream_readable.js:929:16 I have debugged through and I am certain that the file with the correct name exists in the bucket.

I'm really stumped on this so any help would be greatly appreciated.

I have tried writing the file to my local file system also rather than in the response and got the same error.

A temporary (or permanent) workaround that is working is this:

bucket.getSignedUrl({
                action: 'read',
                expires: Math.round(Date.now() / 1000) + (60 * 60 * 24), // 1 day.
                resource: file.name
            }, function (err, url) {
                res.redirect(url);

            });

The workaround works fine but doesn't help me figure out why the createReadStream fails since it shows that the file name and bucket are correct.

I think more code may be helpful to track down the issue. The error being returned is saying the file doesn't exist, which could mean 1) you're asking for the file from the wrong bucket, 2) maybe the file didn't exist at the time you tried to read it, or 3) the file name is simply incorrect (maybe something simple like .jpg vs .jpeg). — Stephen, Oct 13 '14 at 14:37
@Stephen, Thanks, I added some code to the description that shows a workaround. It does look like the filename and bucket are correct. I have tested the same values over and over with different techniques. This is a gist that shows the code I'm testing with: https://gist.github.com/ruairitobrien/6dd587ae8e8b14a2ce5c — Ruairi O'Brien, Oct 13 '14 at 16:02
Thanks for adding more. What's important now is when these functions (createFile and getFile) are being called. If getFile is called before createFile has completed uploading the file, it will be too soon, therefore the file wouldn't exist. Just in case this is useful, I have (likely) a similar app written that uses streams to upload/download with a (not particularly nice looking) web interface: https://github.com/stephenplusplus/sillycloud — Stephen, Oct 13 '14 at 16:11
I am sure the file already exists. No doubt some silly little bug I'm not seeing. I'll keep at it and will post back if I figure it out. Thanks for the sample code! It's really helpful. — Ruairi O'Brien, Oct 13 '14 at 16:26
Sorry I couldn't be of more help -- if you do find out it's a bug with the library, please make a new issue at https://github.com/GoogleCloudPlatform/gcloud-node/ -- also, if you're feeling adventurous, I have a PR on the way that changes the Storage api: https://github.com/GoogleCloudPlatform/gcloud-node/pull/253 - feel free to test it out and see if the problem magically goes away :) — Stephen, Oct 13 '14 at 16:46

score 1 · Accepted Answer · answered May 10 '15 at 14:51

I think you might be using an outdated version of gcloud-node. Here's how I would do an upload and a download with the latest version (as of today, this is v0.14.0).

var fs = require('fs');
var gcloud = require('gcloud');

var project = gcloud({
  projectId: 'your-project-id',
  // keyFilename: '/path/to/your/keyfile.json'
});

var bucket = project.storage().bucket('your-bucket-name');

var filename = 'local-file-name';
var file = bucket.file(filename); // This is a gcloud.File

var getFiles = function() {
  bucket.getFiles(function(err, files) {
    console.log('Files were', files);
  });
};

var createFile = function() {
  fs.createReadStream(filename)
    .pipe(file.createWriteStream())
    .on('error', function (err) {
      console.log('Error!', err);
    })
    .on('complete', function(result) {
      console.log('Complete!', result);
    });
};

var getFile = function() {
  file.createReadStream()
    .pipe(fs.createWriteStream(filename + '.downloaded'))
    .on('error', function(err) {
      console.log('Error downloading!', err);
    });
}

createFile();
getFile();

Running this script should upload a file (local-file-name) on your bucket (your-bucket-name) and then download that file back as local-file-name.download).

Some references to read through (there should be examples on each of these pages):

That's right. I got it working with the newer version recently. I should have updated this question but forgot. Thank you! — Ruairi O'Brien, May 10 '15 at 22:31

Not found error in bucket.createReadStream

1 Answers1