2

I have a problem whit query/scan operation in DynamoDb. I'm using the dynamodb-data-mapper whit aws-data-mapper-annotations in order to do operations whit the database. I have a Lambda function which have to get all the items from a table (in this case I have a table of tags) and I want to paginate the result, whit a 'limit' and a 'start key' specified by the client, because the tags table could contain hundred of thousands records. So I write something like this:

import dbContext from '@dbModel/dbContext';
import Tag from '@dbModel/tables/tags';
import { ScanOptions } from '@aws/dynamodb-data-mapper';

const scanTag = async (filter: any): Promise<{
    items: Tag[],
    lastKey: Partial<Tag>
}> => {
    let items: Tag[] = [];
    let lastKey: Partial<Tag>;
    const dbFilter: ScanOptions = {
        limit: filter.limit,
        startKey: {
            id: filter?.startKey,
        },
        readConsistency: 'strong'
    };

    // use paginator because of the scan limit of 1MB
    const paginator = dbContext.scan(Tag, dbFilter).pages();

    for await (const page of paginator) {
        items = items.concat(page);
        lastKey = paginator.lastEvaluatedKey;
    }

    return Promise.resolve({
        items: items,
        lastKey: lastKey
    });
};

export default scanTag;

This is the problem: the scan operations returns a non-sorted list of items, so how I can ensure that the second call by the client (so the client has already received the first 'n' items and he want to get the next 'm' items) will return a distinct set of items? For example: the first call return the result: [T5, T8, T1, T4], I want to do a second call providing the key of T4 as startKey, and I want to be sure the there's no T5, T8, T1 or T4 in my second response.

Thank you all in advance!

Simone Urbani
  • 130
  • 1
  • 5

0 Answers0