papaparse too fast for database angular 5

Question

I'm using papaparse to read the csv file to fetch the records but the issue is that it is too fast for the database. It reads the csv file in an instant but the individual record is still being processed in the database API, so only some records are processed asynchronously in the database with random sequence and all the records do not enter the database because of it. My code is

onSubmit() {
this.papa.parse(this.csvFile, {
  step: (row) => {
    var jsonObj = this.arrayToJSON(row.data[0]);
    console.log(jsonObj);
    this.apiService.addEmployees(jsonObj).
      subscribe(
        add => this.arrayToJSON(jsonObj),
        error => console.log("Error :: " + error))
  }
});}

I want all that the papaparse waits for the db request to process, then fetch the next record.

score 0 · Answer 1 · answered Apr 10 '18 at 17:29

You can use Observable.concat for that:

onSubmit() {
  // First of all, prepare an array of observables
  const databaseWrites = [];

  this.papa.parse(this.csvFile, {
    step: (row) => {
      let jsonObj = this.arrayToJSON(row.data[0]);
      console.log(jsonObj);
      databaseWrites.push(this.apiService.addEmployees(jsonObj));
    }
  });

  // Once everything is prepared, use concat to do each operation one after another
  Observable.concat(...databaseWrites).subscribe(() => { console.log('write success !')});
}

Because Observables are cold, you can prepare your requests and subscribe to the concat in order to have each operation being executed one after another.

papaparse too fast for database angular 5

1 Answers1