How to Delete Multiple Items from a DynamoDB Table in Go

Question

I've a DynamoDB table that contains items like this:

type AuthEntry struct {
    UserID    string    `dynamodbav:"userId"`
    Token     string    `dynamodbav:"token"`
    CreatedOn time.Time `dynamodbav:"createdOn"`
}

I need to delete all the AuthEntry items older than 5 minutes (CreatedOn < now - 5 mins) and without a token (Token is empty). It is clear to me how to remove one item at a time... but I'm wondering how to delete multiple items in one shot. Thank u very much.

maybe a duplicate of https://stackoverflow.com/questions/62022594/can-we-query-and-delete-item-in-aws-dynamodb-at-the-same-time/62023976#62023976 — reda la, Mar 10 '21 at 09:56
can you explain further? As I know, aws-sdk-go-v2 mainly enhances modularity... Seth has a good and similar response. — reda la, Mar 10 '21 at 18:28

score 4 · Answer 1 · answered Mar 10 '21 at 21:45

I was looking for an example like the one here below... and I hope it helps other newbie like me. For instance, first I use Scan to retrieve the expired entries, and then I run BatchWriteItemInput to actually delete them.

import (
    "context"
    "time"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb/types"

)

var tableName = "USER_AUTH"

...

type AuthRepository struct {
    ctx context.Context
    svc *dynamodb.Client
}

...

func NewAuthRepository(ctx context.Context) (*AuthRepository, error) {
    cfg, err := config.LoadDefaultConfig(ctx)
    if err != nil {
        return nil, err
    }

    return &AuthRepository{ctx, dynamodb.NewFromConfig(cfg)}, nil
}

...

func (r *AuthRepository) Collect(maxAge int) (int32, error) {
    t := time.Now().Add(time.Duration(maxAge*-1) * time.Millisecond).UTC()
    params := &dynamodb.ScanInput{
        TableName:            aws.String(tableName),
        ProjectionExpression: aws.String("userId"),
        ExpressionAttributeValues: map[string]types.AttributeValue{
            "t": &types.AttributeValueMemberS{*aws.String(t.String())},
        },
        FilterExpression: aws.String("createdOn < :t"),
    }

    result, err := r.svc.Scan(r.ctx, params)
    if err != nil {
        return 0, err
    }

    wr := make([]types.WriteRequest, result.Count)
    for _, v := range result.Items {
        authEntry := &AuthEntry{}
        if err := attributevalue.UnmarshalMap(v, &authEntry); err != nil {
        return 0, err
        }
        wr = append(wr, types.WriteRequest{
            DeleteRequest: &types.DeleteRequest{
                Key: map[string]types.AttributeValue{
                    "userId": &types.AttributeValueMemberS{*aws.String(authEntry.UserID)},
                },
            }})
    }

    input := &dynamodb.BatchWriteItemInput{
        RequestItems: map[string][]types.WriteRequest{
            tableName: wr,
        },
    }

    _, err = r.svc.BatchWriteItem(r.ctx, input)
    return result.Count, nil
}

Cheers for this, got me at a good place to start. One thing to keep in mind: batch operations have a limit of 25 writes. — alex067, Jul 01 '23 at 13:53

score 3 · Answer 2 · answered Mar 10 '21 at 14:50

When it comes to deletion, you have a few options.

deleteItem - Deletes a single item in a table by primary key.

batchWriteItem - The BatchWriteItem operation puts or deletes multiple items in one or more tables. A single call to BatchWriteItem can write up to 16 MB of data, which can comprise as many as 25 put or delete requests

TimeToLive - You can utilize DynamoDBs Time To Live (TTL) feature to delete items you no longer need. Keep in mind that TTL only marks your items for deletion and actual deletion could take up to 48 hours.

I'm not sure which items in your table are part of the primary key, so it's difficult to give you an example. However, this operation is the preferred method to delete multiple items at a time.

How to Delete Multiple Items from a DynamoDB Table in Go

2 Answers2