AWS json output parsing

Question

I am trying to parse a json out of AWS query. I am trying to get the RedrivePolicy.deadLetterTargetArn

So, if I just do:

--output text --query "Attributes.RedrivePolicy"

Output: {"deadLetterTargetArn":"arn:aws:sqs:us-east-2:xxxxxx:cloud-us-east-2-deadletter","maxReceiveCount":4} which is right

when I go to next level:

--output text --query "Attributes.RedrivePolicy.deadLetterTargetArn"

It says None in output.

Please find my json herein:

{
    "Attributes": {
        "ApproximateNumberOfMessagesNotVisible": "0", 
        "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"custodian-notify-subscription\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"SQS:SendMessage\",\"Resource\":\"arn:aws:sqs:us-east-2:xxxxxxxx:cloud-us-east-2\",\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"arn:aws:sns:us-east-2:xxxxxxxxx:cloud-us-east-2-notify\"}}}]}", 
        "MessageRetentionPeriod": "86400", 
        "ApproximateNumberOfMessagesDelayed": "0", 
        "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-2:xxxxxxxxx:cloud-us-east-2-deadletter\",\"maxReceiveCount\":4}", 
        "MaximumMessageSize": "262144", 
        "CreatedTimestamp": "1561753144", 
        "ApproximateNumberOfMessages": "0", 
        "ReceiveMessageWaitTimeSeconds": "10", 
        "DelaySeconds": "0", 
        "KmsDataKeyReusePeriodSeconds": "300", 
        "VisibilityTimeout": "600", 
        "LastModifiedTimestamp": "1561753144", 
        "QueueArn": "arn:aws:sqs:us-east-2:xxxxxxxx:cloud-us-east-2"
    }
}

Answer 1

Quick Answer (TL;DR)

• A JMESPath query on a JSON string will not produce the expected result if you try to run a query against it using the --query keyword in AWS CLI
• In the example provided, the JMESPath query does not work as expected, because the JSON element being queried is technically a JSON string (aka scalar) and not a JSON object (aka dictionary)

Detailed Answer

• The key to understanding why the example is not producing the desired outcome is in distinguishing a JSON string element from a JSON object element.

• We provide some examples below with an extremely simplified JSON scenario that helps illustrate what is going on here.

Example 01

• In this example, we use JMESPath to query JSON. We get the expected result because we are querying against a JSON object element.
• A more detailed description: We have a top-level json object with a single name-value pair.
  • The name in the name-value pair consists of the string "Message"
  • The value in the name-value pair consists of a JSON object
    • Digging deeper, the JSON object has two name-value pairs

        // Original JSON
        {
            "Message": {"Greeting":"Hello","Audience":"World!"}
        }

        // JMESPath query 
        --query 'Message.Greeting'

        // JMESPath result
        'Hello'

Example 02

• In this example, we use JMESPath to query JSON. We get an unwanted result because we are querying against a JSON string element.

  • A more detailed description: We have a top-level json object with a single name-value pair.
    • The name in the name-value pair consists of the string "Message"
    • The value in the name-value pair consists of a JSON string
      • Digging deeper, the JSON string looks similar to a JSON object, but it is not a JSON object, because it follows the conventions of a JSON string

        // Original JSON
        {
            "Message": "{\"Greeting\":\"Hello\",\"Audience\":\"World!\"}"
        }

        // JMESPath query 
        --query 'Message.Greeting'

        // JMESPath result
        None (null result)

See also

• JSON dot org
• related JSON question on StackOverflow
  • related answer