How do I conditionally insert an item into a dynamodb table using boto3

Question

If I have a table with a hash key of userId and a range key of productId how do I put an item into that table only if it doesn't already exist using boto3's dynamodb bindings?

The normal call to put_item looks like this

table.put_item(Item={'userId': 1, 'productId': 2})

My call with a ConditionExpression looks like this:

table.put_item(
    Item={'userId': 1, 'productId': 2}, 
    ConditionExpression='userId <> :uid AND productId <> :pid', 
    ExpressionAttributeValues={':uid': 1, ':pid': 3}
)

But this raises a ConditionalCheckFailedException every time. Whether an item exists with the same productId or not.

score 90 · Accepted Answer · edited Oct 12 '16 at 01:47

90

The documentation for this unfortunately isn't super clear. I needed to accomplish something similar, and here's what worked for me, using boto3:

try:
    table.put_item(
        Item={
            'foo':1,
            'bar':2,
        },
        ConditionExpression='attribute_not_exists(foo) AND attribute_not_exists(bar)'
    )
except botocore.exceptions.ClientError as e:
    # Ignore the ConditionalCheckFailedException, bubble up
    # other exceptions.
    if e.response['Error']['Code'] != 'ConditionalCheckFailedException':
        raise

Similar to the other answer, the key is in the attribute_not_exists function, but it was unclear to me initially how to get that to work. After some experimentation, I was able to get it going with the above.

edited Oct 12 '16 at 01:47

Kristian

21,204
19
101
176

answered Sep 14 '16 at 19:54

jimjkelly

1,631
14
15

11

If you add this import `from boto3.dynamodb.conditions import Attr` then the `ConditionExpression` can be one of `ConditionExpression=Attr("foo").ne(1) & Attr("bar").ne(2)` or `ConditionExpression = Attr("foo").not_exists() & Attr("bar").not_exists()` I find the name of `not_exists()` confusing. It reads like it is going to check if the attribute is missing completely from the record, but in fact it checks equality inferred from the `Item` value. docs for `Key, Attr` https://boto3.amazonaws.com/v1/documentation/api/latest/reference/customizations/dynamodb.html#ref-dynamodb-conditions – Davos Nov 30 '18 at 00:58
2

"I find the name of not_exists() confusing." - that's because you don't understand it. It really does check for the existence of an attribute, but it can be used to check for a matching partition key. How? Well think of the case when you are using put_item to add a record with a partition key value that matches an existing record. DynamoDB is about to overwrite it (default behavior), but then your Condition says not if it already has the partition key (which is does as all records must have the partition key attribute). It's the overwriting of the existing record where the equality come from – aaa90210 May 07 '19 at 04:29
This approach works only when foo and bar are hash/range keys. It won't work for regular attributes. See https://forums.aws.amazon.com/thread.jspa?messageID=582356#582356 – Big Pumpkin Nov 06 '19 at 20:53
Supporting this answer with an example from AWS: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html#Expressions.ConditionExpressions.PreventingOverwrites – Matt Jan 17 '20 at 22:22
1

This is misleading. Answer is missing important part - what partition and sort key is. Assume that one of foo or bar must be partition key then one of conditions `attribute_not_exists(foo)` `attribute_not_exists(bar)` is enough. – Ivan Tichy Nov 16 '22 at 19:08

score 13 · Answer 2 · answered Nov 15 '18 at 12:59

13

You dont need the sortkey( or range key) just the partition key or hash key is enough.

try:
    table.put_item(
        Item={
            'foo':1,
            'bar':2,
        },
        ConditionExpression='attribute_not_exists(foo)'
    )
except botocore.exceptions.ClientError as e:
    # Ignore the ConditionalCheckFailedException, bubble up
    # other exceptions.
    if e.response['Error']['Code'] != 'ConditionalCheckFailedException':
        raise

answered Nov 15 '18 at 12:59

Venkat.V.S

349
3
7

2

This is incorrect. The table has a compound key which requires both the hash key and the range (sort) key to uniquely identify a record. This will fail the ConditionExpression if other records exist sharing the same hash_key e.g. `{ 'foo': 1, 'bar': 1}, { 'foo': 1, 'bar': 3}, { 'foo': 1, 'bar': 4}` which is not what the OP wants, it should only fail if `{ 'foo': 1, 'bar': 3}` exists. It might suit other use case, or perhaps your hash keys alone are enough to uniquely identify records, but if that is the case then your range key is doing nothing, sorting a bag of 1 is meaningless. – Davos Nov 30 '18 at 00:46
OK I take it back I tried it and this does work! Sorry. I think this is because a ConditionExpression is meant to be applied to Attributes, not Keys. The record to put is identified by the `Item` being passed in, and of course either of the Key attributes will always exist by definition. – Davos Nov 30 '18 at 02:40
3

yes, Thats fine. Even i was confused at the beginning.. hey just out of curiosity how come wrong answers get upvoted and the right ones are not getting that traction. why? – Venkat.V.S Dec 15 '18 at 15:23
It's not always the case on stack overflow. Mostly good answers are upvoted but there are varying opinions on what is good. Sometimes answers were right in the past but are now wrong. Sometimes people just prefer one style over another. What's not nice is when people downvote without providing a comment as to why. There's no perfect forum when humans get involved, always politics. Most people are nice though, this is a good discussion of it: https://codeblog.jonskeet.uk/2018/03/17/stack-overflow-culture/ – Davos Dec 17 '18 at 01:01
The problem here is that we do not know what partition (and sort if there is one) key is. If foo is partition key then `attribute_not_exists(foo)` is enough. – Ivan Tichy Nov 16 '22 at 19:05

Eyal Ch · Answer 3 · 2016-05-05T16:12:31.930

3

i think that you will get better documentation using client.put_item rather than table.put_item

from boto3 documentation:

To prevent a new item from replacing an existing item, use a conditional expression that contains the attribute_not_exists function with the name of the attribute being used as the partition key for the table. Since every record must contain that attribute, the attribute_not_exists function will only succeed if no matching item exists.

ConditionExpression:

ConditionExpression (string) -- A condition that must be satisfied in order for a conditional PutItem operation to succeed.

An expression can contain any of the following:

Functions: attribute_exists | attribute_not_exists | attribute_type | contains | begins_with | size These function names are case-sensitive.

i am using dynamodb2 overwrite parameter on item.save()

edited May 05 '16 at 16:12

answered May 05 '16 at 14:59

Eyal Ch

9,552
5
44
54

Well... I have read the documentation. But I've found writing a condition expression to be non-intuitive and I'm worried that the way I've written it might not be doing what I think it's doing. So was wondering if someone else had actually done it before. Do you have an example of how you've done it? I'll update my answer showing how I've done it – aychedee May 05 '16 at 15:56
i added to the answer how i do it.. (using overwrite) – Eyal Ch May 05 '16 at 16:13
Ah okay... that's using boto. I'm asking about boto3. But thanks for taking the time to have a look. – aychedee May 05 '16 at 16:49
1

Generally when using boto3 it is good advice to use the (higher level) `resource` API where functionality exists, and if not then fall back to using the `client` API. In this case - in this case `table.put_item` is definitely the better option, because you can use `from boto3.dynamodb.conditions import Key, Attr` and those two things make building expressions much easier. The docs are simpler for `resource` because there's fewer options, but most of what it says in the `client` docs also applies to `resource`. – Davos Nov 30 '18 at 00:21

score 0 · Answer 4 · answered Apr 04 '23 at 16:11

To prevent a new item from replacing an existing item, use a conditional expression that contains the attribute_not_exists function with the name of the attribute being used as the partition key for the table.

You can use the errorfactory which will create an error class based on the codes returned.

import boto3

res = boto3.resource('dynamodb')
table = res.Table('mytable') 

try: 
    conditionalUpdateResponse = table.put_item(
        Item={'userId': 1, 'productId': 2},
        ConditionExpression='attribute_not_exists(userId)',
    )
except res.meta.client.exceptions.ConditionalCheckFailedException as e: 
    print(e)

Source: https://github.com/boto/boto3/issues/2235#issuecomment-574929483

AWS Boto3 docs: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/client/put_item.html

How do I conditionally insert an item into a dynamodb table using boto3

4 Answers4

Linked