84

I'm trying to use the update_item functionality for DynamoDB in boto3.

I'm struggling right now to update lists for items. I would like to create a new list if the list does not exist yet and otherwise append to the existing list.

Using an UpdateExpression of the form SET my_list = list_append(my_list, :my_value) returns an error "The provided expression refers to an attribute that does not exist in the item" if the list does not exist yet.

Any idea how I would have to modify my UpdateExpression?

Anton Menshov
  • 2,266
  • 14
  • 34
  • 55
fabian
  • 883
  • 1
  • 6
  • 6

2 Answers2

190

You can use list_append(if_not_exists()) construction.

UpdateExpression:

'SET my_list2 = list_append(if_not_exists(my_list2, :empty_list), :my_value)'

ExpressionAttributeValues:

{ ":my_value":{"L": [{"S":"test"}]}, ":empty_list":{"L":[]} }

Update: as mentioned in the comments, boto3 now raises an error for the expression above and a version without explicit types works: { ":my_value": ["test"], ":empty_list":[] }.

Borys Serebrov
  • 15,636
  • 2
  • 38
  • 54
  • Thanks for the hint! Pretty neat solution. – fabian Jan 26 '16 at 11:17
  • 1
    Do you have any idea if this is subject to race-conditions? I'm looking over the AWS documentation, and they don't seem to have any locking/transactions (at least for the Python SDK). It certainly seems like this code could a: read the list from the DB, b: append the new value, and c: overwrite the old value, which _could_ have been changed in the meantime. – John C Sep 26 '17 at 17:06
  • 2
    I can't find a direct confirmation in the documentation, but I would expect single UpdateItem operation to be atomic, the closest thing I found in faq - "You can increment or decrement a numeric attribute in a row using a single API call. Similarly, you can atomically add or remove to sets, lists, or maps", see https://aws.amazon.com/dynamodb/faqs/ – Borys Serebrov Sep 29 '17 at 15:37
  • Hi I am struggling to reproduce the answer. `{ "TableName": "WishboxUsers", "Key": { "uid": { "S": "$context.requestId" } }, "UpdateExpression": "SET my_list2 = list_append(if_not_exists(my_list2, :empty_list), :my_value)", "ExpressionAttributeValues": { ":my_value":[{"S":"test"}], ":empty_list":[] } }` Can you help me? Are there different versions of the DynamoDB API? How can I change them? I get the response: `"Unrecognized collection type class com.amazonaws.dynamodb.v20120810.AttributeValue"` – csikos.balint Oct 22 '17 at 16:33
  • 1
    @csikos.balint I think the problem is not with the expression, but with something else - I've just tired with DynamoDB local and the example works, complete set of parameters: `{ TableName: 'my_table', Key: { my_key: 'hash1' }, UpdateExpression: 'SET my_list2 = list_append(if_not_exists(my_list2, :empty_list), :my_value)', ExpressionAttributeValues: { ":my_value":[{"S":"test"}], ":empty_list":[] } };`. Initially I have "my_table" defined with one "my_key" hash field. So it's better if you post your problem as a separate question with complete code example. – Borys Serebrov Oct 23 '17 at 21:39
  • 2
    @aidinism I think you misunderstood the question and the answer. It is not about preventing the duplicate values, the question was "how to create a new list if the list does not exist yet and otherwise append to the existing list.". – Borys Serebrov Feb 28 '18 at 15:23
  • 2
    thank you very much for your suggestion! I got it working for my project here if anyone interested: https://github.com/networth-app/networth/blob/f79f2c8d55b2614a09465331cf9b360c31d0bb1f/api/lib/dynamodb.go#L155 – Kien Pham Sep 11 '18 at 06:46
  • 6
    I had to set my `ExpressionAttributeValues` to `{":my_value": {"L": [{"S": "test"}]}, ":empty_list": {"L": []}}` for anyone that runs into the same problem as me. – ubomb Dec 19 '18 at 17:54
  • for me "':empty_list': []" was sufficient (no need for "L") – hormberg Sep 05 '19 at 13:34
  • 1
    Perhaps something has changed since this answer was given, but using Python/Flask and this code would throw exceptions about me trying to pass a map to list_append. I think boto3 dynamically determines the type now? Anyway. Changing the ExpressionAttributeValues to `{ ":my_value": ["test"], ":empty_list":[] }` fixed the problem for me. – Tom Apr 25 '21 at 10:54
  • 1
    @Tom thanks! The other example did not work for me and caused type errors. The pure brackets did the trick. Maybe that is the way python/boto3 expects it. – Andreas Nov 01 '22 at 02:55
8

An alternative to Boris solution could be to use set instead of list datatype and use the ADD keyword, it does exactly what you want.

With Add, the update expression becomes: ADD setName :s

And the expression attribute values can be like: {":s": {"SS":["First", "Second"]}}

http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.ADD

Ayush Pateria
  • 620
  • 12
  • 22
  • 2
    Note that "The order of the values within a set is not preserved" so if you want an `ordered collection` then you want a List. Your expression example is misleading because `["First", "Second"]` could easily be returned as `["Second", "First"]`. The benefit of set is that it contains no duplicates. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html#HowItWorks.DataTypes – Davos Nov 08 '18 at 05:59
  • 1
    I like this solution. Just adding note that the `set` type can only contain scalar elements (so no sets of `map` type) – codecitrus Nov 27 '20 at 23:52