DynamoDB doesn't have an auto-increment, but below two features should help you implement what you want using the below DynamoDB features:
- DynamoDB supports transactions where multiple tables can be simultaneously updated atomically
- DynamoDB also supports conditional expressions which will make sure we can stop an update based on a condition
Below is a simple idea which uses the above features
Let's call your table DATA
You can maintain another COUNTER table which stores the missing keys and the highestId assigned until now (Something like below where the value is of Document type)
Key Value RecordVersionNumber
+-------------+-----------------------------------------------------+--------+
| Counter | {"Highestid": 7, "MissingKeys": ["3,4,6"]} | 10 |
+-------------+-----------------------------------------------------+--------+
DELETE would add an ID to the missingKeys if it's deleted from the data table
INSERT could choose one of the missing keys in which case it removes the element from missing keys
If missingKeys is empty, INSERT would edit the HighestId instead
- Check the DynamoDB column size limits and design your table accordingly for your data
- Counter need not be a table specifically, you could even use one of the rows of your DATA table. I used a separate table for clarification
- Take a look at the transaction isolation levels
For your version of auto-increment to work below steps should happen atomically during each operation:
During a DELETE:
- WRITE into DATA table: Delete the record with given id if it exists using a conditional delete
- READ from COUNTER table: Read existing missing keys
- WRITE into COUNTER table: Add the deleted Id to missingKeys, sort the list, increase the RecordVersionNumber by 1 and perform a conditional update on RecordVersionNumber
Reads (2) are done in the java code, writes are prepared as an TransactWriteItem statement
If (1) fails because the key doesn't exist, the other steps will not happen
In (3), to avoid parallel calls corrupting our data, we must add a conditional check to make sure to write only if RecordVersionNumber of existing record is same as the one we read in (2)
i.e. (3) will fail with ConditionalCheckFailed exception if any other INSERT/DELETE operation has edited the Counter after we read it in (2)
The ConditionalCheckFailed operation needs to be handled and whole method: (1), (2), (3) should be retried few times before returning a failure response
During an INSERT:
- READ from the COUNTER table: Read existing missing keys, HighestId
- WRITE into DATA table: Do a Conditional put for the new "Id-Name" record. If missingKeys is empty, id=HighestId+1 else Id=missingKeys.first
- WRITE into COUNTER table: Based on what's done in step(2), remove the missingKeys value which was assigned or increment the highestId, write back the record with a conditional update on RecordVersionNumber
If two INSERT calls happen parallely, both could read the same COUNTER data and try to write conflicting id into the DATA table.
In such cases one of the inserts will fail at (2) due to ConditionalCheckFailed error and the steps (1 to 3) should be retried
If the COUNTER table is modified due to a DELETE after we read missing keys in (1), then step(3) will fail and whole of (1-3) should be retried in code
The number of retries due to parallel calls should be reduced if you are able to queue the number of INSERT/DELETE operations or control the number of parallel executions
