Error in getting JSON serializable output from Lambda from DynamoDB

Question

I have a lambda function in python where I want to return all the contents of a DynamoDB table. I will be using this output in a GET API of AWS API Gateway, so the output needs to be ready accordingly. My lambda now looks like this:

import boto3
import json

def lambda_handler(event, context):
    client = boto3.resource("dynamodb")
    table = client.Table("User")
    response = table.scan()['Items']
    
    print(json.dumps(response))
     
    return json.dumps(response)

The output is

Response:
{
  "errorMessage": "Object of type Binary is not JSON serializable",
  "errorType": "TypeError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 14, in lambda_handler\n    print(json.dumps(response))\n",
    "  File \"/var/lang/lib/python3.7/json/__init__.py\", line 231, in dumps\n    return _default_encoder.encode(obj)\n",
    "  File \"/var/lang/lib/python3.7/json/encoder.py\", line 199, in encode\n    chunks = self.iterencode(o, _one_shot=True)\n",
    "  File \"/var/lang/lib/python3.7/json/encoder.py\", line 257, in iterencode\n    return _iterencode(o, 0)\n",
    "  File \"/var/lang/lib/python3.7/json/encoder.py\", line 179, in default\n    raise TypeError(f'Object of type {o.__class__.__name__} '\n"
  ]
}

How to solve this?

Answer 1

The only immediate issue with you code is that you are only retrieving the first page of the scan, see the documentation to learn how scan works. Before returning the result, you need to check if LastEvaluatedKey is present in the response; see also the answer to this question.

To return all the items in your table, your code should be updated as follows:

import boto3
import json


client = boto3.resource('dynamodb')

def lambda_handler(event, context):
    
    table = client.Table('User')
    response = table.scan()
    data = response['Items']

    while response.get('LastEvaluatedKey'):
        response = table.scan(ExclusiveStartKey=response['LastEvaluatedKey'])
        data.extend(response['Items'])

    return json.dumps(data)

To debug the specific error, you need to inspect the content of the response from DynamoDB. You can check the response syntax documentation to know what you should expect.