Javascript, spliced FileReader for large files with Promises, how?

Question

I'm trying to use FileReader to read several files sequentially using promises for that. The problem is that I need to divide the reading operations in several chunks to make them doable. By doing so, I lose the Promise chain. This is what happen in the following code, where I get the console log here, then catched (meaning I've lost the chain), then inside and then finished. Somehow the Promise in upload is not respected.

Here it is the code (please go to the last EDIT, I keep the original text nonetheless)

var reader = new FileReader();
reader.onloadend = function(e) {
  if (e.target.readyState == 2) {
    console.log('inside')
    start = temp_end;
    temp_end = start + BYTES_PER_CHUNK;
    if (temp_end > end) temp_end = end;
    upload();
  }
};

Array.reduce(function(promise, item) {
  start = 0;
  temp_end = start + BYTES_PER_CHUNK;
  end = parseInt(totalsize);
  if (temp_end > end) temp_end = end;
  uploading_file = item;
  return upload()
  .then(function(){
    console.log('not shown');
  })
  .catch(console.log('catched'));
},0);

function upload() {
  return new Promise(function(resolve,reject) {
    if (start < end) {
      console.log('here')
      var chunk = uploading_file.slice(start, temp_end);
      reader.readAsBinaryString(chunk);
    } else {
      console.log('finished')
      uploading_file = null;
      resolve();
    }
  }
}

EDIT1: new code I'm trying, still the console.log stop 1 appears before the upload and the here 2

        var start = 0;
        var temp_end = start + BYTES_PER_CHUNK;
        var end = parseInt(item.size);
        if (temp_end > end) temp_end = end;
        uploading_file = item;
        console.log('here 1')
        Promise.resolve().then(function() {
            return upload();
        })
        .then(function(){
            console.log('here 2')
        })
        .catch(console.log('stop 1'));

        function upload() {
            console.log('upload')
            if (start < end) {
                var chunk = uploading_file.slice(start, temp_end);
                var reader = new FileReader();
                reader.readAsArrayBuffer(chunk);
                reader.onload = function(e) {
                    if (e.target.readyState == 2) {
                        start = temp_end;
                        temp_end = start + BYTES_PER_CHUNK;
                        if (temp_end > end) temp_end = end;
                        return upload();
                    }
                }
            } else {
                console.log('finished')
                uploading_file = null;
                return Promise.resolve();
            }
         }

EDIT2: The catch logs were there due to a lack of function(). The following code seems to work, but I cannot see the value content that I am interested at the end.

The problem is that promises are indeed not working here, the console.log says

here 1
here 2
here 4
here 3

The code:

        var item; // item is a file
        var BYTES_PER_CHUNK = 100000;
        var start = 0;
        var temp_end = start + BYTES_PER_CHUNK;
        var end = parseInt(item.size);
        if (temp_end > end) temp_end = end;
        var content = ''; // to be filled by the content of the file
        var uploading_file = item;
console.log('here 1');
        Promise.resolve().then(function() {
console.log('here 2');
            return upload();
        })
        .then(function(content){
console.log('here 4');
            console.log(content); // this is empty (!!)
        })
        .catch(function(){
            console.log('stop 1')
        });

        function upload() {
            if (start < end) {
                var chunk = uploading_file.slice(start, temp_end);
                var reader = new FileReader();
                reader.readAsArrayBuffer(chunk);
                reader.onload = function(e) {
                    if (e.target.readyState == 2) {
                        content += new TextDecoder("utf-8").decode(e.target.result);
                        start = temp_end;
                        temp_end = start + BYTES_PER_CHUNK;
                        if (temp_end > end) temp_end = end;
                        return upload();
                    }
                }
            } else {
                uploading_file = null;
                console.log(content); // it shows the content of the file
console.log('here 3');
                return Promise.resolve(content);
            }
        }

I'm finding it really hard to understand what you're trying to do. You have a lot of undeclared variables, shared state, and incorrect code (`Array.reduce`? Is `Array` actually a variable in your real code?). And `upload` seems to have nothing to do with uploading and the returned promise sometimes doesn't resolve or reject. I'll take a stab at guessing the attempt, but if you could clarify exactly what the purpose of the code is. — Jacob, Aug 07 '16 at 18:30
Please go to the last EDIT for the final code. In any case, the Array.reduce was meant to mean that I had an array and that I used the reduce, but this part of the code is indeed not important for the problem I want to isolate. I hope with the last code it is clearer what I want to achieve? The final goal is to read files by chunks and to keep the control of this through promises — GWorking, Aug 07 '16 at 18:42
@GWorking is that worked, seems this guy needs the same solution like your's https://stackoverflow.com/questions/57206375/multifile-upload-as-chunks — Jayavel, Jul 26 '19 at 10:15
@Jayavel I've dropped a link to this question, I haven't been working with file uploads since ... 2 years ago, a little bit rusty down here :) — GWorking, Aug 03 '19 at 05:31

score 1 · Accepted Answer · answered Aug 07 '16 at 18:48

Your upload() function does not always return a Promise. It does in the else condition, but the if condition doesn't. You have a return statement, but it's inside of a callback, so it won't be received by the caller of upload.

Try changing that to this:

function upload() {
  if (start < end) {
    return new Promise(function (resolve, reject) {
      var chunk = uploading_file.slice(start, temp_end);
      var reader = new FileReader();
      reader.readAsArrayBuffer(chunk);
      reader.onload = function(e) {
        if (e.target.readyState == 2) {
          content += new TextDecoder("utf-8").decode(e.target.result);
          start = temp_end;
          temp_end = start + BYTES_PER_CHUNK;
          if (temp_end > end) temp_end = end;
          resolve(upload());
        }
      }
    });
  } else {
    uploading_file = null;
    console.log(content); // it shows the content of the file
    return Promise.resolve(content);
  }
}

Now, upload always returns a Promise instead of undefined.

Fantastic. This is it. Thousand thanks! I am troubling now by using a Array.reduce here. Since this has been nicely solved, I'm opening a new question for that, http://stackoverflow.com/questions/38822187/javascript-uploading-several-files-within-an-array-reduce-with-promises-how, in case you have time — GWorking, Aug 08 '16 at 06:02

Javascript, spliced FileReader for large files with Promises, how?

1 Answers1

Linked