How to get the row count of a table instantly in DynamoDB?

Question

I'm using boto.dynamodb2, and it seems I can use Table.query_count(). However it had raised an exception when no query filter is applied.

What can I do to fix this?

BTW, where is the document of filters that boto.dynamodb2.table.Table.Query can use? I tried searching for it but found nothing.

score 31 · Accepted Answer · answered Jul 15 '15 at 22:00

31

There are two ways you can get a row count in DynamoDB.

The first is performing a full table scan and counting the rows as you go. For a table of any reasonable size this is generally a horrible idea as it will consume all of your provisioned read throughput.

The other way is to use the Describe Table request to get an estimate of the number of rows in the table. This will return instantly, but will only be updated periodically per the AWS documentation.

The number of items in the specified index. DynamoDB updates this value approximately every six hours. Recent changes might not be reflected in this value.

answered Jul 15 '15 at 22:00

JaredHatfield

6,381
2
29
32

Can I force AWS to update the row number index as my app won't write frequently. – Kane Blueriver Jul 16 '15 at 02:21
1

No. AWS will update the row count every 6 hours automatically for you. If you want a more accurate row count you could attempt to keep track of the number of rows outside of the table, such as in another DynamoDB table, but that introduces its own set of issues. – JaredHatfield Jul 16 '15 at 02:23
AWS won't update this even if I just have had scanned the whole table? – Kane Blueriver Jul 16 '15 at 02:26
2

No. This value will be updated by AWS periodically and at the moment there is no way to force any type of update to it. – JaredHatfield Jul 17 '15 at 20:12
1

how can it be done with some condition? must a table scan be used then? – cryanbhu Sep 13 '18 at 02:17
3

You can definitely force it from the console. Go to the overview tab of your table and click "Manage Live Count" and then click "Start Scan." @cryanbhu – Joshua Wolff Jul 06 '19 at 18:08
4

Estimate? wth? It is one of the simplest things possible in most databases to get a count of all rows. if dynamodb doesn't make that simple then it is worthless and no sane person should use it. – Samantha Atkins Jul 29 '19 at 04:28
5

@SamanthaAtkins Remember that DynamoDb is a No-SQL data store. It was designed to solve different problems than those solved by relational databases. I can assure you that sane people designed it and sane people use it and find great value with it. – Brian Boyd Mar 12 '21 at 20:12
@BrianBoyd - I assure you that there are tons of people who are using it even though it's completely incorrect for them. I wouldn't be surprised if it turns out the majority of people using DynamoDB should actually be using a relational DB. There's not a lot of scenarios out there where being quick is actually more useful than being correct. – ArtOfWarfare Aug 14 '23 at 19:47

score 24 · Answer 2 · answered Mar 17 '17 at 16:49

24

As per documentation boto3

"The number of items in the specified table. DynamoDB updates this value approximately every six hours. Recent changes might not be reflected in this value."

import boto3

dynamoDBResource = boto3.resource('dynamodb')
table = dynamoDBResource.Table('tableName')
print(table.item_count)

or you can use DescribeTable:

import boto3

dynamoDBClient = boto3.client('dynamodb')
table = dynamoDBClient.describe_table(
    TableName='tableName'
)
print(table)

answered Mar 17 '17 at 16:49

Strabek

2,391
3
32
39

3

Note that DynamoDB.Table.item_count updates approximately every **six hours**. Recent changes might not be reflected in this value. Refer to doc: [link](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html?highlight=creation_date_time#DynamoDB.Table.item_count) – Black Cat Jun 07 '22 at 18:18

score 9 · Answer 3 · answered Nov 07 '17 at 17:45

9

If you want to count the number of items:

import boto3

client = boto3.client('dynamodb','us-east-1')
response = client.describe_table(TableName='test')
print(response['Table']['ItemCount'])

#ItemCount (integer) --The number of items in the specified table.
# DynamoDB updates this value approximately every six hours.
# Recent changes might not be reflected in this value.

Ref: Boto3 Documentation (under ItemCount in describe_table())

answered Nov 07 '17 at 17:45

Hari K

251
3
8

1

Note - this may not show accurate result for recent tables, refer [here](https://stackoverflow.com/a/31441922/8101287) – Ankit Agrawal Jan 06 '21 at 08:03

theBuzzyCoder · Answer 4 · 2017-03-14T11:02:06.363

You can use this, to get count of entire table items

from boto.dynamodb2.table import Table

dynamodb_table = Table('Users')
dynamodb_table.count()  # updated roughly 6 hours

Refer here: http://boto.cloudhackers.com/en/latest/ref/dynamodb2.html#module-boto.dynamodb2.table

query_count method will return the item count based on the indexes you provide.

For example,

from boto.dynamodb2.table import Table

dynamodb_table = Table('Users')
print dynamodb_table.query_count(
    index='first_name-last_name-index',  # Get indexes from indexes tab in dynamodb console
    first_name__eq='John',  # add __eq to your index name for specific search
    last_name__eq='Smith'  # This is your range key
)

You can add the primary index or global secondary indexes along with range keys. possible comparison operators

__eq for equal

__lt for less than

__gt for greater than

__gte for greater than or equal

__lte for less than or equal

__between for between

__beginswith for begins with

Example for between

print dynamodb_table.query_count(
    index='first_name-last_name-index',  # Get indexes from indexes tab in dynamodb console
    first_name__eq='John',  # add __eq to your index name for specific search
    age__between=[30, 50]  # This is your range key
)

How to get the row count of a table instantly in DynamoDB?

4 Answers4

Linked