How to extract properties from my state machine output when using EventBridge Input Transformers

Question

I'm having trouble passing one of the values returned by my AWS state machine as input to a Lambda using the EventBridge service.

I created a state machine in AWS Step Functions to model a specific problem in our domain. Once the state machine finishes, I want to perform another operation from inside one of my Lambdas. To make that work, I created a new rule using EventBridge: whenever the state machine finishes, it triggers my lambda with a specific Json input.

My problem is in how to extract properties from the state machine output and pass them as properties of the lambda input.

Say my state machine returns this:

{
    "usefulObject":{
        "usefulProperty":"value"
    },
    "anotherProperty":"anotherValue"
}

I want to receive the following payload in my lambda:

{
    "property":"value"
}

Initially, I thought I would be able to do this using the "Input Transformation" option on the EventBridge rule, something like:

• InputTransformer Path:

{"propertyValue":"$.usefulObject.usefulProperty"}

• InputTransformer Template:

{"property":<propertyValue>}

However, during testing, I realized that the event payload contains a lot more data than my state machine output. In reality, the state machine output is wrapped in this "event container" object, something like this:

{
    "version": "0",
    "id": "...",
    "detail-type": "Step Functions Execution Status Change",
    "source": "aws.states",
    "account": "...",
    "time": "2020-11-10T13:59:57Z",
    "region": "us-east-1",
    "resources": [
        "...myStateMachineArn..."
    ],
    "detail": {
        "executionArn": "...myStateMachineExecutionArn...",
        "stateMachineArn": "...myStateMachineArn...",
        "name": "ff72036a-2917-c657-80e7-2589b7b76d59",
        "status": "SUCCEEDED",
        "startDate": 1605016794597,
        "stopDate": 1605016797936,
        "input": "{\n  \"usefulObject\":{\n    \"usefulProperty\": \"value\"\n  },\n  \"anotherProperty\": \"anotherValue\"\n}",
        "inputDetails": {
            "included": true
        },
        "output": "{\n  \"usefulObject\":{\n    \"usefulProperty\": \"value\"\n  },\n  \"anotherProperty\": \"anotherValue\"\n}",
        "outputDetails": {
            "included": true
        }
    }
}

As you can see, in the actual event payload, my statemachine data is stored as a stringified Json value inside the output node. If I then change my Input Transformation path to something like:

{"propertyValue":"$.detail.output.usefulObject.usefulProperty"}

I get an empty result for property in the transformed input. Turns out, JsonPath is unable to traverse the stringified value as part of the Json payload and will fail on the search.

How can I extract the usefulProperty value out of that Json string in the event payload so that I can pass it to my lambda function? Is there a way to do it using purely JsonPath that I'm missing? Perhaps there is a way to configure AWS to not convert the payload into a string and just make it part of the whole event payload? Any other alternatives?

Answer 1

There are a couple of things you can try to keep your input and output from being stringified by the interpreter:

1. Instead of input :

{"propertyValue":"$.usefulObject.usefulProperty"}

try

{"propertyValue.$":"$.usefulObject.usefulProperty"}

or

{"propertyValue": $.usefulObject.usefulProperty}

2. Make sure your lambda does not return a stringified version of the result i.e: return JSON.stringify(usefulObject)

3. If none of these help you can try converting the output string in your step function template with:

{
    "propertyValue.$": "States.StringToJson($.detail.output.usefulObject.usefulProperty)"
}

4. longshot: make sure you are not deliberately converting it to string, in your step function template with States.JsonToString somewhere... :)

as per the docs on https://states-language.net/

5. Try pulling out the value from the event object within the Lambda, and then use it:

exports.lambdaHandler = async (event) => {
  // parse your input or output string and use it as an object;
  let receivedParams = JSON.parse(event.detail.input);
  let propertyValueObject = {
    propertyValue: receivedParams.usefulObject.usefulProperty,
  };

  // do something in your lambda with propertValueObject 
  // which is equal to: { propertyValue: 'value' }
  console.log(propertyValueObject);
};

let event = {
    "version": "0",
    "id": "...",
    "detail-type": "Step Functions Execution Status Change",
    "source": "aws.states",
    "account": "...",
    "time": "2020-11-10T13:59:57Z",
    "region": "us-east-1",
    "resources": [
        "...myStateMachineArn..."
    ],
    "detail": {
        "executionArn": "...myStateMachineExecutionArn...",
        "stateMachineArn": "...myStateMachineArn...",
        "name": "ff72036a-2917-c657-80e7-2589b7b76d59",
        "status": "SUCCEEDED",
        "startDate": 1605016794597,
        "stopDate": 1605016797936,
        "input": "{\n  \"usefulObject\":{\n    \"usefulProperty\": \"value\"\n  },\n  \"anotherProperty\": \"anotherValue\"\n}",
        "inputDetails": {
            "included": true
        },
        "output": "{\n  \"usefulObject\":{\n    \"usefulProperty\": \"value\"\n  },\n  \"anotherProperty\": \"anotherValue\"\n}",
        "outputDetails": {
            "included": true
        }
    }
}

const lambdaHandler = (event) => {
  // parse your input or output string and use it as an object;
  let receivedParams = JSON.parse(event.detail.input);
  let propertyValueObject = {
    propertyValue: receivedParams.usefulObject.usefulProperty,
  };

  // do something in your lambda with propertValueObject 
  // which is equal to: { propertyValue: 'value' }
  console.log(propertyValueObject);
};

lambdaHandler(event)