Dynamo db pagination

Question

I want to use pagination in dynamodb using aws-sdk DocumentClient() I am using node.js.

What I want to do is get first 10 items and then return these value to the user. After that user makes a new request in which he tell the server to start from 10 and the server get other 10 from 10 to 20 and return the response back. I have tried the LastEvaluatedKey But my scenario is different. Is there any way that I can tell dynamodb to start from specific Item e.g 1 and then set Limit: 10.

Dynamodb pagination is not skip and limit type. You have to give lastevalekey to client and ask client to forward it to get the next page — Nidhin David, May 10 '19 at 09:49
thank you. Is there any way to get all the Items using query to get all the records from the collection. I donnt want to use scan. — Nouman Dilshad, May 10 '19 at 09:54
Nope...DynamoDB returns max of 1MB data in a single query response. — Nidhin David, May 10 '19 at 09:57

Nouman Dilshad · Accepted Answer · 2019-05-10T11:45:36.597

I found a way to work around this. You need to get the LastEvaluatedKey from the dynamodb response and send it back to the front-end then your front-end should send the LastEvaluatedKey in params and you can use it as ExclusiveStartKey.

getItems(pageSize, lastItem?) {
    try {
      const params = {
        TableName: 'User',
        Limit: pageSize,
      };
      if (lastItem) {
        params.ExclusiveStartKey = { item_id: lastItem};
      }
      const response = await dynamoDb.scan(params).promise();
      return {
         items: response.Items,
         lastItem: response.LastEvaluatedKey
      }

    } catch (error) {
      throw error;
    }

score 0 · Answer 2 · answered Apr 29 '22 at 18:22

Using Limit and LastEvaluatedKey for pagination is a common practice with DynamoDB BUT it's important to notice that Limit isn't the number of records returned on your result set; rather this is the number of records analyzed on a query operation.

Have a look at the AWS documentation on Queries and the Limit property: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#API_Query_RequestSyntax

The maximum number of items to evaluate (not necessarily the number of matching items). If DynamoDB processes the number of items up to the limit while processing the results, it stops the operation and returns the matching values up to that point, and a key in LastEvaluatedKey to apply in a subsequent operation, so that you can pick up where you left off. Also, if the processed dataset size exceeds 1 MB before DynamoDB reaches this limit, it stops the operation and returns the matching values up to the limit, and a key in LastEvaluatedKey to apply in a subsequent operation to continue the operation. For more information, see Query and Scan in the Amazon DynamoDB Developer Guide.

While this is a generally applied workaround to pagination it may lead to unexpected results on large datasets.

Dynamo db pagination

2 Answers2

Linked