0

If anyone here is familiar with the node-cloudfiles module for node.js, I could use some help in several different areas. Unfortunately, is seems the authors are nearly impossible to reach via their github repo (EDIT: nevermind, someone did reach out to me, I'll send an update when I have an answer of some sort prepared.)

I'll start with my most basic challenge: is there a way to track the progress of the upload? I have tried many things, but the object returned from the .addFile command does not seem to hold any sort of progress stats.

Here is a basic outline of what I am working with.

var readStream = fs.createReadStream(path+'.'+extension, streamopts);
var upOpts = {
    headers: {
        'content-type': 'video/'+extension,
        'content-length': totalBytes
    },
    remote: CDNfilename, 
    stream: readStream
};

//reqStream is the object returned from the 'request' module, 
//which is used by the 'cloudfiles' module.
var reqStream = cloudClient.addFile(Container.name, upOpts, function (err, uploaded) {
    if (err) { console.log(err); }

});

At first I thought I could just use the .bytesWritten property connected to an interval timer, but the object is not a normal node writeStream, so there is no such property.

agentOOjoe
  • 85
  • 1
  • 6

2 Answers2

2

Charlie (the author of the module) told me that this is possible because it's using a pipe and you just check the data events from the object returned from .addFile, like so:

reqStream.on('data', function () { 
  /* track progress /* 
});

Whenever you need to contact somebody from the nodejitsu team, join the #nodejitsu channel on IRC, they're really active.

alessioalex
  • 62,577
  • 16
  • 155
  • 122
  • Hey alessioalex, thanks for the response. I probably should have mentioned that I tried to track the data listener for the original read stream (readStream), but it turns out that it does not get paused, as I expected (since it is a .pipe). Because of this, the original readStream ends as soon as the system can tear through it, and does not actually represent the data received by the stream going to cloudfiles. – agentOOjoe Dec 21 '11 at 18:54
  • Now, I tried a data listener on the reqStream, and I am very surprise to see that it even has a data listener (i had assumed it was more like a traditional node writeStream, which don't have data listeners). However, it is behaving sort-of strangely. It only emits this event one time, and the data.length is always way too small. It looks like it is only emitted once, when the upload is complete. I'll keep looking at it though. – agentOOjoe Dec 21 '11 at 18:58
0

At the time of writing this answer, there isn't really a good way to get upload progress for files being sent to cloudfiles. However, one of the nodejitsu geniuses implemented chunked uploading, which in my case, eliminates the need for progress reports. Thanks Bradley.

agentOOjoe
  • 85
  • 1
  • 6