0

Scenario

I need a background upload queue that sends files to a server. The queue should send the files in sequential order as they are pushed into the queue (FIFO).

My solution

var pendingFiles = [];
var filesOperation = null;

uploadNextAsync = function(file) {
  var next;
  if (!pendingFiles.length) {
    return WinJS.Promise.as();
  }
  next = pendingFiles.shift();
  fileslogger.debug("Uploading " + next);
  return fileQuery.folder.getFileAsync(next).then(function(file) {
    return Server.sendFileAsync(file).then(function() {
      return filesOk += 1;
    }, function(error) {
      filesNok += 1;
      return logger.error(error.message, error);
    }).then(function() {
      if (pendingFiles.length) {
        return uploadNextAsync(inspection);
      }
    });
  });
};

createTaskForFile = function(file) {
  if (pendingFiles.length == 0) {
    pendingFiles = [file.name]
    filesOperation = uploadNextAsync(file);
  } else {
    pendingFiles.push(file.name);
    return filesOperation.then(function() {
      return uploadNextAsync(file);
    });
  }
};

Problem

It seems that sometimes if createTaskForFile is called very quickly in succession then 2 files end up being uploaded at the same time. So somewhere is a little glitch either in the createTastForFile function on how it uses the fileOperation.then construct or inside the uploadNextAsync does something wrong?

philk
  • 2,009
  • 1
  • 23
  • 36

1 Answers1

1

Your problem is that pendingFiles is always empty. In createTaskForFile, you would set it to an one-element array then, but immediately call uploadNextAsync() which shifts it out. I guess your script might work if you shifted the file after the file has been uploaded.

However, you actually don't need this array. You can just queue the action to filesOperation, which would be a promise representing the upload of all current files.

var filesOperation = WinJS.Promise.as();
function createTaskForFile(file) {
  return filesOperation = filesOperation.then(function() {
    return uploadNextAsync(file);
  });
}

function uploadAsync(next) {
  fileslogger.debug("Uploading " + next.name);
  return fileQuery.folder.getFileAsync(next.name).then(function(file) {
    return Server.sendFileAsync(file);
  }).then(function() {
    return filesOk += 1;
  }, function(error) {
    filesNok += 1;
    return logger.error(error.message, error);
  });
}
Bergi
  • 630,263
  • 148
  • 957
  • 1,375
  • aha! Interesting yes. How does this work in regard of memory consumption. Are the "done" promises kept around? And what about a break condition that cancels the update queue like a file with a specific type (.xml) that has been uploaded? Should I set the `filesOperation` to `WinJS.Promise.as()` again in that case? – philk Dec 05 '14 at 16:17
  • The done promises are not kept around once they are resolved. Only the last one is stored in `filesOperation` (and subsequent ones overwrite this variable). You might store them however somewhere else, as `createTaskForFile` returns each of them. – Bergi Dec 06 '14 at 13:42
  • I'm not sure what you mean by "break". Yes, if one `uploadAsync` promise is rejected, it would cancel all queued operations (without the possibility to restart them). However, as it is written `uploadAsync` never rejects, all errors are handled (with `filesNok += 1`). To create a new (empty) queue you'd then reset `filesOperation` to `WinJS.Promise.as()`; if you wanted to resume the queue you'd need to actually handle the exception with a "wait-until-continue" promise. – Bergi Dec 06 '14 at 13:48