Downloading and merging multipart file in javascript

Question

I need some help with below requirement.

Download huge multipart csv file from webserver and join those parts at client side before opening it for user.
I can get the number of parts and have their URLs from webserver before downloading.
I need to do it in client browser hence i think javascript is the best choice.
Users use IE as primary browser but I cannot ask them to change their security settings (vbscript/activxcontrol will not work)

"Download huge multipart csv file from webserver and join those parts at client side before opening it for user" is pretty much a reason not to do this. If the user has to download a huge data set, your solution is not the right solution. Make the client ask the server for only the bits the user can actually see, and write your code to deal with that instead of needing all the data in one giant whopping go. — Mike 'Pomax' Kamermans, Feb 23 '13 at 16:34
I cannot agree with you more. But the application is already there. Today users manually merge the files. I am just trying to automate that part. — user2102619, Feb 23 '13 at 18:11

score 0 · Answer 1 · answered Feb 23 '13 at 20:48

If you have no choice but to work with this, you could use an aggregation:

var data; // will contain the retrieved data

/**
 * Grab data from the first URL in the list, adding it
 * to the global data variable. If the list is empty,
 * we got all the data and we succeed. If an XHR fails, we fail.
 */
function prepData(urlList, success, fail) {
  if(urlList.length===0) {
    succes(); // wrap in timeout if you need to break out of call chain
    return;
  }
  var xhr = new XMLHttpRequest();
  var url = urlList.splice(0,1)[0];
  xhr.open("GET", url, true);
  xhr.onreadystatechange = function() {
    if (xhr.status === 200 && xhr.readyState === 4) {
      data += xhr.responseText;
      prepData(urlList, success, fail);
    } else {
      fail(urlList, xhr);  // again, wrap in timeout to break out of call chain
    }
  }
}

We can then call this with code like:

/**
 * prepare our data prior to application "start"
 */
prepData(['...','...','...'], function (){
  console.log("finished aggregating data");
  // all went well. Start the actual data-consuming code
}, function(notLoadedList, failedXhrObject){
  console.log("ohnoes!", notLoadedList, failedXhrObject);
  // something went wrong, and we can fail gracefully
});

Downloading and merging multipart file in javascript

1 Answers1