How do I query AWS DynamoDB in python?

Question

I'm fairly new to NoSQL and using AWS DynamoDB. I'm calling it from AWS Lambda using python 2.7 I'm trying to retrieve a value from an order_number field.
This is what my table looks like(only have one record.):

primary partition key: subscription_id

and my secondary global index: order_number

Is my setup correct? If so given the order_number how do I retrieve the record using python?
I can't figure out the syntax to do it.

I've tried

response = table.get_item( Key = {'order_number': myordernumber} )

But i get:
An error occurred (ValidationException) when calling the GetItem operation: The provided key element does not match the schema: ClientError

score 34 · Accepted Answer · edited Mar 17 '19 at 02:33

DynamoDB does not automatically index all of the fields of your object. By default you can define a hash key (subscription_id in your case) and, optionally, a range key and those will be indexed. So, you could do this:

response = table.get_item(Key={'subscription_id': mysubid})

and it will work as expected. However, if you want to retrieve an item based on order_number you would have to use a scan operation which looks through all items in your table to find the one(s) with the correct value. This is a very expensive operation. Or you could create a Global Secondary Index in your table that uses order_number as the primary key. If you did that and called the new index order_number-index you could then query for objects that match a specific order number like this:

from boto3.dynamodb.conditions import Key, Attr

response = table.query(
    IndexName='order_number-index',
    KeyConditionExpression=Key('order_number').eq(myordernumber))

DynamoDB is an very fast, scalable, and efficient database but it does require a lot of thought about what fields you might want to search on and how to do that efficiently.

The good news is that now you can add GSI's to an existing table. Previously you would have had to delete your table and start all over again.

thanks for your help, i tried the query code you gave me and its giving me a `global name 'Key' is not defined` and its referring to that line of code you gave me `KeyConditionExpression=Key('order_number').eq(myordernumber))` — MalcolmInTheCenter, Dec 09 '15 at 19:20
This is what my function looks like `def getRecord( self, table_name, indexName, gsiKey, gsiValue): table = self.dynamodb.Table( table_name ) response = table.query(IndexName=indexName,KeyConditionExpression=Key(gsiKey).eq(gsiValue))` — MalcolmInTheCenter, Dec 09 '15 at 19:50
Nevermind I figured it out, I was only importing boto3 but needed to do `from boto3.dynamodb.conditions import Key, Attr ` — MalcolmInTheCenter, Dec 09 '15 at 21:49
^^ . Would be great if this comment is included in the answer :) — Dawny33, Apr 21 '17 at 07:19

score 16 · Answer 2 · answered Jul 16 '16 at 00:03

Make sure you've imported this:

from boto3.dynamodb.conditions import Key, Attr

If you don't have it, you'll get the error for sure. It's in the documentation examples.

Thanks @altoids for the comment above as this is the correct answer for me. I wanted to bring attention to it with a "formal" answer.

score 9 · Answer 3 · edited Aug 05 '20 at 18:53

9

To query dynamodb using Index with filter:

import boto3
from boto3.dynamodb.conditions import Key, Attr

dynamodb = boto3.resource('dynamodb', region_name=region)
table = dynamodb.Table('<TableName>')

response = table.query(
    IndexName='<Index>',
    KeyConditionExpression=Key('<key1>').eq('<value>') & Key('<key2>').eq('<value>'),
    FilterExpression=Attr('<attr>').eq('<value>')
)

print(response['Items'])

If filter is not rquired then don't use FilterExpression in query.

edited Aug 05 '20 at 18:53

vahdet

6,357
9
51
106

answered Jul 21 '20 at 10:22

singh30

1,335
17
22

Thanks for showing me the `&` operator syntax in the `KeyConditionExpression` – matusko Aug 26 '22 at 14:03

score 0 · Answer 4 · answered Jun 23 '21 at 23:33

import boto3
from boto3.dynamodb.conditions import Key
dynamodb = boto3.resource('dynamodb', region_name=region_name)
table = dynamodb.Table(tableName)

def queryDynamo(pk, sk):
    response = table.query(
        ProjectionExpression="#pk, #sk, keyA, keyB",
        ExpressionAttributeNames={"#pk": "pk", "#sk": "sk"},
        KeyConditionExpression=
            Key('pk').eq(pk) & Key('sk').eq(sk)
    )
    return response['Items']

score 0 · Answer 5 · answered Aug 26 '22 at 14:28

If you use the boto3 dynamodb client, you can do the following (again you would need to use subscription_id as that is the primary key):

dynamodb = boto3.client('dynamodb')
response = dynamodb.query(
    TableName='recurring_charges', 
    KeyConditionExpression="subscription_id = :subscription_id",
    ExpressionAttributeValues={":subscription_id": {"S": "id"}}
)

afelaho · Answer 6 · 2023-01-26T22:35:14.153

So far, this is the cleanest way I've discovered; the query is in JSON format.

dynamodb_client = boto3.client('dynamodb')

def query_items():
    arguments = {
        "TableName": "your_dynamodb_table",
        "IndexName": "order_number-index",
        "KeyConditionExpression": "order_number = :V1",
        "ExpressionAttributeValues": {":V1": {"S": "value"}},
    }
    return dynamodb_client.query(**arguments)

How do I query AWS DynamoDB in python?

6 Answers6

Linked