Using an EventBridge event pattern string in a lambda function

Question

I have a lambda function using Python. It's connected to an EventBridge rule that triggers every time there's a change in a Glue table.

The event pattern it's outputting looks something like this:

{
    "version":"0",
    "detail":{
        "databaseName":"flights-db",
        "typeOfChange":"UpdateTable",
        "tableName":"flightscsv"
    }
}

I want to get the tableName and databaseName values from this output into the function as a variable.

My Lambda function:

import json
import boto3

def lambda_handler(event, context):
    boto3_version = boto3.__version__

    return_statement = 'Boto3 version: ', boto3_version,\
                       'Event output: ', event

    return {
        'statusCode': 200,
        'body': json.dumps(return_statement)
    }

I was expecting to get the event pattern output from the event in my return statement but that's not the case.

When testing this function the return output for event is:

{\"key1\": \"value1\", \"key2\": \"value2\", \"key3\": \"value3\"}

This key and values are defined like this in the Test Pattern for the function.

The eventbridge rule is defined like this:

How can I get the values from the event pattern to a variable? Do I need to configure the test pattern to get the results into event?

EDIT: Picture of log events for the table change event:

What is the outcome of `print(event)` when the function is triggered. Also can you show your EB rule that triggers the lambda? — Marcin, Feb 09 '21 at 08:19
Thanks, but I mean the `print(event)` when your function is actually invoked by EB, not when you test it in console. The `event` objects will be different in both cases. — Marcin, Feb 09 '21 at 08:55
I see. The rule is active and I'm triggering it with a table change but don't know where I can see this `print(event)` information. Do you know where I can check it? Maybe CloudWatch? — ire, Feb 09 '21 at 09:10
CloudWatch Logs. I also found the events documented [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types). I think this is enough to try to provide an answer. — Marcin, Feb 09 '21 at 09:10

Marcin · Accepted Answer · 2021-02-09T09:50:06.203

The event object generated by CloudWatch (CW) Events / Event Bridge (EB) are listed here. These events will be passed to your function when it is going to get triggered by EB.

Your EB Event Pattern should be:

{
  "source": ["aws.glue"],
  "detail-type": ["Glue Data Catalog Table State Change"]
}

The above should match changes to any tables in your glue catalog. The event should be similar to the one below:

{
    "version": "0",
    "id": "2617428d-715f-edef-70b8-d210da0317a0",
    "detail-type": "Glue Data Catalog Table State Change",
    "source": "aws.glue",
    "account": "123456789012",
    "time": "2019-01-16T18:16:01Z",
    "region": "eu-west-1",
    "resources": [
        "arn:aws:glue:eu-west-1:123456789012:table/d1/t1"
    ],
    "detail": {
        "databaseName": "d1",
        "changedPartitions": [
            "[C.pdf, dir3]",
            "[D.doc, dir4]"
        ],
        "typeOfChange": "BatchCreatePartition",
        "tableName": "t1"
    }
}

Thus, to get tableName and databaseName your lambda function could be:

import json
import boto3

def lambda_handler(event, context):
    boto3_version = boto3.__version__

    print(event)

    table_name = event['detail']['tableName']
    database_name = event['detail']['databaseName']

    print(table_name, database_name)

    return_statement = {
        'boto3_version': boto3_version,
        'table_name': table_name,
        'database_name': database_name
    }

    return {
        'statusCode': 200,
        'body': json.dumps(return_statement)
    }

For testing, you can setup sample EB event in your lambda test window:

Thank you. I'm still having trouble locating where in CloudWatch I can see the `print(event)`. I located my event (table change) under Cloudwatch > Log Groups but I can't see the information. I'll update the post with the picture of log events for the table change event. — ire, Feb 09 '21 at 09:39
@ire CloudWatch Logs. There should be a `log group` for your function. If its not there, its either not triggered or your lambda execution role is missing CW permissions. Can you check that? — Marcin, Feb 09 '21 at 09:42
I attached the picture of log events in the main post. It's very possible that it's an issue with permissions. Will look into that now and let you know. — ire, Feb 09 '21 at 09:45
@ire I also updated the answer with how to use sample EB event for Glue as test in your lambda console. Its easier to test your function with this `event`. — Marcin, Feb 09 '21 at 09:51
Got it working on my usecase :) I'll give the reward once it lets me — ire, Feb 09 '21 at 12:39

Using an EventBridge event pattern string in a lambda function

1 Answers1