JQ parsing of escaped quotes

Question

We have an AWS autoscaling lifecycle hook that is placing messages on an SQS queue. We are trying to parse these with JQ, and while the format is mostly JSON, all the key/value pairs within the body object have escaped double quotes. Can't seem to find an AWS option to not escape the quotes, or a JQ option to successfully parse them. Example here:

    {
    "Messages": [
        {
            "Body": "{
\"Origin\":\"AutoScalingGroup\",
\"LifecycleHookName\":\"termination_lifecycle_hook\",
\"Destination\":\"EC2\",
\"AccountId\":\"<redacted>\",
\"RequestId\":\"65e1eaeb-1bde-42ef-93b5-ac14987a2848\",
\"LifecycleTransition\":\"autoscaling:EC2_INSTANCE_TERMINATING\",
\"AutoScalingGroupName\":\"test-um-prod-20230822043811258400000005\",
\"Service\":\"AWS Auto Scaling\",
\"Time\":\"2023-08-29T20:42:36.581Z\",
\"EC2InstanceId\":\"i-02ca28b1a415a4d48\",
\"LifecycleActionToken\":\"1e7f23b9-4a9a-4cad-9051-6039d815e9b1\"
}",
            "ReceiptHandle": "AQEBN8E7SCNS44lemWqlfMVvOl9EfGo7Yb5b+VhF5jXlw2i41pwQJFvfaAFwBEquO8SOsGyaIUfFiE9qEUQoQOgyUoz5a+Rx/yD9jq2qfz/gRcSwhfCAPHyBvhjJdIGmlyKtNnTzuL183Wti4BO4G+Z7bLzn/WiIT+JrSfn/VofI34HnB0w0Om3UYTnd6OoXuCu2WxzasyMeFcImuCYdt+5+tMzauEtsL7EVdSZwpW3S4oatkuxfzzyxHsP4X0IIKt6fwIzHzCRhi9kmo5QnZAQCnjHo+RBNZXWU9wbk3SToE3nCdLJhbups5SOHAUHLO1PwdVb7RvKMmYYdHfzoTQBLeWJ1w7MEMrhANC9uOu7cVzS0hAAc9qPLzzoV1TkIBn/0Q597cl6dNyfkWpPQEOKtUg==", 
            "MD5OfBody": "fa2bc63f28d96c532288b09831a8b73f", 
            "MessageId": "4c5b0c2c-9085-4bde-8de3-49ae3f63afd4"
        }
    ]
}

As you can see, all objects except "Body" are fine, but all the key/values within "Body" are escaped.

The `Body` string does not have linebreaks in it, does it? That would make your input invalid JSON. — Bergi, Aug 30 '23 at 04:59
It's not that the quotes are escaped, but that the `Body` key contains a JSON object serialized as _string_ — knittl, Aug 30 '23 at 06:05

score 1 · Accepted Answer · answered Aug 30 '23 at 04:58

1

You can use the fromjson function for parsing JSON-encoded values. In your case, that would be

jq '.Messages | .[] | .Body | fromjson'

answered Aug 30 '23 at 04:58

Bergi

630,263
148
957
1,375

Will try that... I did see the "fromjson" in the docs, but since the entire purpose of JQ is to parse from JSON, it wasn't clear what this was for. – Matt Aug 30 '23 at 06:05
1

That did the trick, thanks! Since we mainly wanted the instanceId, what worked was: jq -r '.Messages[].Body | fromjson | .EC2InstanceId' – Matt Aug 30 '23 at 06:10

JQ parsing of escaped quotes

1 Answers1