1

I have a table with storage size of 2.7MB. I am trying to get all records using dynamoose scan method, but it is very slow (35 seconds to get 2.7MB)

The code is written in node.js (typescript). First I used scan method and checked for lastKey:

this.ReportsModel.scan().exec(function (err, data, lastKey) {
if(lastKey) { 
    this.ReportsModel.scan().startAt(lastKey).exec(function (err, data, lastKey) {
        return callback(null, data)
    });
}

})

Afterwards I tried scan.all. From Dynamoose API I understood that the default delay between recursive scans is 1 sec. In order to minimize scanning time I tried to minimize the delay time and set the delay to 1 msec and changed the code:

this.ReportsModel.scan().all(1).exec((err, data)

But the scan is still very slow. What can I do to make scan.all to be faster?

student
  • 11
  • 2

1 Answers1

0

I was facing the same slowness and then I tried playing with delay parameter. I was able to get the records faster when I used 0.01 instead of 1 for delay. I have been using it in prod since few weeks now without any issue.

learner
  • 35
  • 1
  • 7