How to convert a DynamoDB Json in a regular JSON in Python?

Question

I get a json from DynamoDB and that is the format:

payload_stack = {'Records': [{'eventID': '123456', 'eventName': 'INSERT', 'eventVersion': '1.1',
                  'eventSource': 'aws:dynamodb', 'awsRegion': 'sa-east-1',
                  'dynamodb': {'ApproximateCreationDateTime': 1644956685.0,
                               'Keys': {'body_field': {'N': '1931'}},
                               'NewImage': {'body_field': {'N': '1931'}, 'txt_vlr_parm_requ': {'M': {
                                   'headers': {'M': {'Authorization': {
                                       'S': 'token'},
                                       'correlationID': {'S': '987654321'}}},
                                   'requestContext': {
                                       'M': {'requestId': {'S': '123'}}},
                                   'body': {'M': {'avro_schema': {
                                       'S': '{"type":"record","namespace":"Tutorialspoint","name":"Employee","fields":[{"name":"Name","type":"string"},{"name":"Age","type":"int"}, {"name":"Address","type":"string"}, {"name":"Role","type":"string"} ]}'},
                                       'cluster': {'S': 'events'}, 'sigla': {'S': 'ft7'},
                                       'subject': {'S': 'teste-dynamo'},
                                       'branch': {'S': 'development'},
                                       'id_requisicao': {'N': '1818'}}}}},
                                            'nom_tabe': {'S': 'tabela_teste'},
                                            'cod_situ_psst_ingo': {'S': 'NOVO'}, 'historic': {
                                       'S': '{"historico": [{"data/hora": "09-02-22 18:18:41", "status": "NOVO"}]}'},
                                            'nom_arqu_bckt': {'S': 'arquivo.avro'}},
                               'SequenceNumber': '87226300000000005691898607', 'SizeBytes': 1672,
                               'StreamViewType': 'NEW_IMAGE'},
                  'eventSourceARN': 'arn:aws'}]}

However I need to convert into a regular json and take only the 'body' field, for example:

'body': {
        "cluster": "events",
        "subject": "teste-dynamo",
        "id_requisition": 1818,
        "branch": "development",
    }

To catch the body field I can imagine how to do, like getting indexes on Python.

But any idea how can I convert this DYNAMODB JSON in a regular JSON?

Thanks.

Might help: [convert single quote json data file to double quote json data file (without mangling inner quotes) · GitHub](https://gist.github.com/mbrzusto/23fe728966247f25f3ec) — John Rotenstein, Mar 15 '22 at 21:23
@JohnRotenstein this didn't work for me, but I found this one (https://github.com/Alonreznik/dynamodb-json) and it helped me a lot. I will put in the question the little code I developed. — Nakano, Mar 16 '22 at 17:33

score 2 · Answer 1 · answered Jun 16 '22 at 09:51

I authored a library called cerealbox that makes it easier to perform this common conversion as follows.

from cerealbox.dynamo import from_dynamodb_json

# convert the DynamoDB image to a regular python dictionary
result = from_dynamodb_json(payload_stack['Records'][0]['dynamodb']['NewImage'])

# access the body as a regular dictionary
body = result['txt_vlr_parm_requ']['body']

The documentation covers how to perfom the inverse using as_dynamodb_json.

This can also be done using boto3's TypeDeserializer/TypeSerializer - a good example of this can be found here

score 0 · Answer 2 · answered Mar 16 '22 at 17:51

I was able to develop a little code to convert this DynamoDB json into a regular json. I used the dynamodb_json import.

##that one in the question
payload_stack = {...} 

convert_regular_json = json.loads(payload_stack)
print(convert_regular_json)

The output:

{
    'Records': [{
        'eventID': '123456',
        'eventName': 'INSERT',
        'eventVersion': '1.1',
        'eventSource': 'aws:dynamodb',
        'awsRegion': 'sa-east-1',
        'dynamodb': {
            'ApproximateCreationDateTime': 1644956685.0,
            'Keys': {
                'body_field': 1931
            },
            'NewImage': {
                'body_field': 1931,
                'txt_vlr_parm_requ': {
                    'headers': {
                        'Authorization': 'token',
                        'correlationID': '987654321'
                    },
                    'requestContext': {
                        'requestId': '123'
                    },
                    'body': {
                        'avro_schema': '{"type":"record","namespace":"Tutorialspoint","name":"Employee","fields":[{"name":"Name","type":"string"},{"name":"Age","type":"int"}, {"name":"Address","type":"string"}, {"name":"Role","type":"string"} ]}',
                        'cluster': 'events',
                        'sigla': 'ft7',
                        'subject': 'teste-dynamo',
                        'branch': 'development',
                        'id_requisicao': 1818
                    }
                },
                'nom_tabe': 'tabela_teste',
                'cod_situ_psst_ingo': 'NOVO',
                'historic': '{"historico": [{"data/hora": "09-02-22 18:18:41", "status": "NOVO"}]}',
                'nom_arqu_bckt': 'arquivo.avro'
            },
            'SequenceNumber': '87226300000000005691898607',
            'SizeBytes': 1672,
            'StreamViewType': 'NEW_IMAGE'
        },
        'eventSourceARN': 'arn:aws'
    }]
}

And to catch the 'body' field:

catch_body_payload = var['Records'][0].get('dynamodb').get('NewImage').get('txt_vlr_parm_requ').get('body')

You can probably just use `catch_body_payload = var['Records'][0]['dynamodb']['NewImage']['txt_vlr_parm_requ']['body']` — John Rotenstein, Mar 16 '22 at 21:26

How to convert a DynamoDB Json in a regular JSON in Python?

2 Answers2