What is the proper way to return data using HTTP-API(v2) + Lambda + DynamoDB as a JSON response?

Question

I used to use the REST API, but since v2 I find myself using it more. Is there a proper way to return data "neatly" other than manipulating the database response before returning? I used to use the model feature with REST (v1). What's the recommended way to do the same here?

Here's an example of what I'm trying to do. I'm selecting specific columns, while avoiding the error:

An error occurred (ValidationException) when calling the UpdateItem operation: Invalid UpdateExpression: Attribute name is a reserved keyword "owner"

(got this for a column named "owner" and "name") and since integers/floats return as a "Decimal":

Object of type Decimal is not JSON serializable

I added the class to set them properly as integers/floats.

import json
import boto3
from decimal import Decimal

class DecimalEncoder(json.JSONEncoder):
  def default(self, obj):
    if isinstance(obj, Decimal):
      return str(obj)
    return json.JSONEncoder.default(self, obj)


def lambda_handler(event, context):
    try:
        dynamodb = boto3.resource('dynamodb')
        table = dynamodb.Table('SomeTable')
        response_body = ''
        status_code = 0
        response = table.scan(
            ProjectionExpression="#col1, #col2, #col3, #col4, #col5",
            ExpressionAttributeNames={
                "#col1": "col1",
                "#col2": "col2",
                "#col3": "col3",
                "#col4": "col4",
                "#col5": "col5"
            }
        )
        items = response["Items"]
        mapped_items = list(map(lambda item: {
            'col1': item['col1'],
            'col2': item['col2'],
            'col3': item['col3'],
            'col4': item['col4'],
            'col5': item['col5'],
        }, items))
        response_body = json.dumps(mapped_items, cls=DecimalEncoder)
        status_code = 200

    except Exception as e:
        response_body = json.dumps(
            {'error': 'Unable to get metadata from SomeTable: ' + str(e)})
        status_code = 403

    json_response = {
        "statusCode": status_code,
        "headers": {
            "Content-Type": "application/json"
        },
        "body": response_body
    }

    return json_response

This just looks too much for a simple "GET" request of some columns in a table

EDIT:

import json
import boto3
from boto3.dynamodb.conditions import Key

dynamodb = boto3.resource('dynamodb')

def lambda_handler(event, context):

    try:
        table = dynamodb.Table("mytable")
        response = table.query(
            ProjectionExpression='col1, col2, col3,col4, col5, col6'
        )
        response = json.loads(json.dumps(response, default=str))
        return {
            "statusCode": 200,
            "body": response
        }

    except Exception as e:
        print(f"Error: {e}")
        return {
            "statusCode": 500,
            "body": "Error: Something went wrong!"
        }

the above function returns the error:

Error: An error occurred (ValidationException) when calling the Query operation: Either the KeyConditions or KeyConditionExpression parameter must be specified in the request.

there is no condition for my case, i am requesting all rows.

score 2 · Accepted Answer · answered Mar 02 '23 at 13:44

2

I know I am not answering your question directly, but the point of having a lambda function in the API - Lambda - DynamoDB pattern is not to make lambda invisible (you can call DynamoDB directly, after all), but to abstract the database call in an efficient manner. For example, having the lambda aware of the structure of the database would save you from making expensive scans and do a more lightweight query instead.

answered Mar 02 '23 at 13:44

Tom Kitt

72
6

Hi thanks for the reply. Can you expand on "having the lambda aware of the structure of the database would save you from making expensive `scan`s" thing? I am currently digging into the `query` option.. – Imnotapotato Mar 02 '23 at 14:08
A `scan` is literally expensive, because it consumes a lot of RCU, and AWS charges you for these. If you `query` the db by partition key (and better still sort key and index key), the database will go over fewer records and AWS will charge you less! By "being aware of the structure" I mean knowing the keys and indexes in the table and using them to minimize the cost of the db query. – Tom Kitt Mar 02 '23 at 14:16
are you sure it's even possible to list all items from the db using `query`? can you helps me write the function (editing my main post with the new function im writing). all examples out there use scan and i think this is why my first example used it (that's what i found at the beggining) – Imnotapotato Mar 05 '23 at 14:05
You are right. If you need to read all records in the table, you are doing a `scan`. – Tom Kitt Mar 13 '23 at 15:25

What is the proper way to return data using HTTP-API(v2) + Lambda + DynamoDB as a JSON response?

1 Answers1