Modelling Complex Types for DynamoDB in Kotlin

Question

I have a DynamoDB table that I need to read/write to. I am trying to create a model for reading and writing from DynamoDB with Kotlin. But I keep encountering com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMappingException: MyModelDB[myMap]; could not unconvert attribute when I run dynamoDBMapper.scanPage(...). Some times myMap will be MyListOfMaps instead, but I guess it's from iterating the keys of a Map.

My code is below:

@DynamoDBTable(tableName = "") // Non-issue, I am assigning the table name in the DynamoDBMapper
data class MyModelDB(

    @DynamoDBHashKey(attributeName = "id")
    var id: String,

    @DynamoDBAttribute(attributeName = "myMap")
    var myMap: MyMap,

    @DynamoDBAttribute(attributeName = "MyListOfMapItems")
    var myListOfMapItems: List<MyMapItem>,
) {
    constructor() : this(id = "", myMap = MyMap(), myListOfMaps = mutableListOf())

    @DynamoDBDocument
    class MyMap {
        @get:DynamoDBAttribute(attributeName = "myMapAttr")
        var myMapAttr: MyMapAttr = MyMapAttr()

        @DynamoDBDocument
        class MyMapAttr {
            @get:DynamoDBAttribute(attributeName = "stringValue")
            var stringValue: String = ""
        }
    }

    @DynamoDBDocument
    class MyMapItem {
        @get:DynamoDBAttribute(attributeName = "myMapItemAttr")
        var myMapItemAttr: String = ""
    }
}

I am using the com.amazonaws:aws-java-sdk-dynamodb:1.11.500 package and my dynamoDBMapper is initialised with DynamoDBMapperConfig.Builder().build() (along with some other configurations).

My question is what am I doing wrong and why? I have also seen that some Java implementations use DynamoDBTypeConverter. Is it better and I should be using that instead?

Any examples would be appreciated!

Answer 1

A couple comments here. First, you are not using the AWS SDK for Kotlin. You are using another SDK and simply writing Kotlin code. Using this SDK, you are not getting full benefits of Kotlin such as support of Coroutines.

The AWS SDK for Kotlin (which does offer full support of Kotlin features) was just released as DEV Preview this week. See the DEV Guide:

Setting up the AWS SDK for Kotlin

However this SDK does not support this mapping as of now. To place items into an Amazon DynamoDB table using the AWS SDK for Kotlin, you need to use:

mutableMapOf<String, AttributeValue>

Full example here.

To map Java Objects to a DynamoDB table, you should look at using the DynamoDbEnhancedClient that is part of AWS SDK for Java V2. See this topic in the AWS SDK for Java V2 Developer Guide:

Mapping items in DynamoDB tables

You can find other example of using the Enhanced Client in the AWS Github repo.

Answer 2

Ok, I eventually got this working thanks to some help. I edited the question slightly after getting a better understanding. Here is how my data class eventually turned out. For Java users, Kotlin compiles to Java, so if you can figure out how the conversion works, the idea should be the same for your use too.

data class MyModelDB(

    @DynamoDBHashKey(attributeName = "id")
    var id: String = "",

    @DynamoDBAttribute(attributeName = "myMap")
    @DynamoDBTypeConverted(converter = MapConverter::class)
    var myMap: Map<String, AttributeValue> = mutableMapOf(),

    @DynamoDBAttribute(attributeName = "myList")
    @DynamoDBTypeConverted(converter = ListConverter::class)
    var myList: List<AttributeItem> = mutableListOf(),
) {
    constructor() : this(id = "", myMap = MyMap(), myList = mutableListOf())
}

class MapConverter : DynamoDBTypeConverter<AttributeValue, Map<String,AttributeValue>> {
    override fun convert(map: Map<String,AttributeValue>>): AttributeValue {
        return AttributeValue().withM(map)
    }

    override fun unconvert(itemMap: AttributeValue?): Map<String,AttributeValue>>? {
        return itemMap?.m
    }

}

class ListConverter : DynamoDBTypeConverter<AttributeValue, List<AttributeValue>> {
    override fun convert(list: List<AttributeValue>): AttributeValue {
        return AttributeValue().withL(list)
    }

    override fun unconvert(itemList: AttributeValue?): List<AttributeValue>? {
        return itemList?.l
    }

}

This would at least let me use my custom converters to get my data out of DynamoDB. I would go on to define a separate data container class for use within my own application, and I created a method to serialize and unserialize between these 2 data objects. This is more of a preference for how you would like to handle the data, but this is it for me.

// For reading and writing to DynamoDB
class MyModelDB {
    ...
    fun toMyModel(): MyModel {
        ...
    }
}

// For use in my application
class MyModel {
    var id: String = ""
    var myMap: CustomObject = CustomObject()
    var myList<CustomObject2> = mutableListOf()

    fun toMyModelDB():MyModelDB {
        ...
    }
}

Finally, we come to the implementation of the 2 toMyModel.*() methods. Let's start with input, this is what my columns looked like:

myMap:

{
    "key1": {
        "M": {
            "subKey1": {
                "S": "some"
            },
            "subKey2": {
                "S": "string"
            }
        }
    },
    "key2": {
        "M": {
            "subKey1": {
                "S": "other"
            },
            "subKey2": {
                "S": "string"
            }
        }
    }
}

myList:

[
    {
        "M": {
            "key1": {
                "S": "some"
            },
            "key2": {
                "S": "string"
            }
        }
    },
    {
        "M": {
            "key1": {
                "S": "some string"
            },
            "key3": {
                "M": {
                    "key4": {
                        "S": "some string"
                    }
                }
            }
        }
    }
]

The trick then is to use com.amazonaws.services.dynamodbv2.model.AttributeValue to convert each field in the JSON. So if I wanted to access the value of subKey2 in key1 field of myMap, I would do something like this:

myModelDB.myMap["key1"]
        ?.m // Null check and get the value of key1, a map
        ?.get("subKey2") // Get the AttributeValue associated with the "subKey2" key
        ?.s // Get the value of "subKey2" as a String

The same applies to myList:

myModelDB.myList.foreach {
        it?.m // Null check and get the map at the current index
        ?.get("key1") // Get the AttributeValue associated with the "key1"
        ...
}

Edit: Doubt this will be much of an issue, but I also updated my DynamoDB dependency to com.amazonaws:aws-java-sdk-dynamodb:1.12.126