Unmarshall DynamoDB JSON

Question

Given some DynamoDB JSON via a DynamoDB NewImage stream event, how do I unmarshall it to regular JSON?

{"updated_at":{"N":"146548182"},"uuid":{"S":"foo"},"status":{"S":"new"}}

Normally I would use AWS.DynamoDB.DocumentClient, however I can't seem to find a generic Marshall/Unmarshall function.

Sidenote: Do I lose anything unmarshalling DynamoDB JSON to JSON and back again?

If others come looking for the python/boto version of this problem, here's the relevant question: https://stackoverflow.com/questions/36558646/how-to-convert-from-dynamodb-wire-protocol-to-native-python-object-manually-with — villasv, Dec 07 '17 at 15:10

giaour · Accepted Answer · 2017-06-15T21:34:19.187

58

You can use the AWS.DynamoDB.Converter.unmarshall function. Calling the following will return { updated_at: 146548182, uuid: 'foo', status: 'new' }:

AWS.DynamoDB.Converter.unmarshall({
    "updated_at":{"N":"146548182"},
    "uuid":{"S":"foo"},
    "status":{"S":"new"}
})

Everything that can be modeled in DynamoDB's marshalled JSON format can be safely translated to and from JS objects.

edited Jun 15 '17 at 21:34

answered Jun 14 '17 at 06:01

giaour

3,878
2
25
27

ARGH.. what an awful API that it requires that M thing `aws.DynamoDB.Converter.output({ 'M': record.dynamodb.NewImage })` – hendry Jun 14 '17 at 09:04
`AWS.DynamoDB.Converter.output` is the part of the DocumentClient that translates [DynamoDB AttributeValue](http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_AttributeValue.html) objects to JSON-style objects. I assumed the `M` was part of the data you got in the stream event. – giaour Jun 14 '17 at 17:29
4

I opened a PR to improve the API by adding a `marshall` function that doesn't require the `M` key, and it was included in version 2.71.0 of the SDK. – giaour Jun 15 '17 at 21:35
1

Worked fine for me as of April 2020 using the AWS SDK Javascript API. Thanks a lot! – alext Apr 20 '20 at 10:44
I know this is an old answer but this is not working for me and it seems related to the fact that a query can return a mix of json that can be decorated or not with datatyping. For exemple this doesn't work : { "Items": [ { "entityType": { "S": "member" } } ], "Count": 1, "ScannedCount": 1 }... will result in : { Items: undefined, Count: undefined, ScannedCount: undefined } .. even array [] alone doesn't work. – Steve S. Aug 18 '21 at 14:20
Well... following my previous comment, I found in the doc API the AWS.DynamoDB.DocumentClient will do the appropriate unmarshalling. Here the complete doc : https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html#get-property – Steve S. Aug 18 '21 at 19:00
@SteveS. The `unmarshall` function operates on string to [`AttributeValue`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_AttributeValue.html) maps. AWS.DynamoDB.DocumentClient knows which fields in JSON responses returned by DynamoDB need to be unmarshalled. In the example you give above, every member of the `Items` array is unmarshalled by the DocumentClient. – giaour Aug 26 '21 at 16:43

score 46 · Answer 2 · answered Feb 03 '21 at 05:31

46

AWS SDK for JavaScript version 3 (V3) provides nice methods for marshalling and unmarshalling DynamoDB records reliably.

const { marshall, unmarshall } = require("@aws-sdk/util-dynamodb");

const dynamo_json = { "updated_at": { "N": "146548182" }, "uuid": { "S": "foo" }, "status": { "S": "new" } };

const to_regular_json = unmarshall(dynamo_json);

const back_to_dynamo_json = marshall(to_regular_json);

Output:

// dynamo_json    
{ 
      updated_at: { N: '146548182' },
      uuid: { S: 'foo' },
      status: { S: 'new' }
}

// to_regular_json
{ updated_at: 146548182, uuid: 'foo', status: 'new' }

// back_to_dynamo_json
{
   updated_at: { N: '146548182' },
   uuid: { S: 'foo' },
   status: { S: 'new' }
}

answered Feb 03 '21 at 05:31

burntsugar

57,360
21
58
81

2

For this to work for me in TypeScript, I had to cast unmarshall(recordImage as { [key: string]: AttributeValue }); // AttributeValue = import { AttributeValue } from "@aws-sdk/client-dynamodb"; – Mike Dubs Jan 19 '22 at 22:12
5

Wow! You have NO IDEA how hard this is to find.. I don't see this documented much if at all or I am blind in the DynamoDB docs and this seems like a must have tool when working with it! – Sandy Garrido Aug 23 '22 at 21:27

Unmarshall DynamoDB JSON

2 Answers2

Linked