5

I am attempting to use jq to parse a AWS CloudFront configuration JSON file and change a few values so that I can then issue an update statement with that configuration.

The Document

The document format snippet that matters, with redacted values, is:

{
    "ETag": "REDACTED",
    "DistributionConfig": {
        "Origins": {
            "Quantity": 2,
            "Items": [
                {
                    "Id": "redacted-1",
                    "DomainName": "redacted1.us-east-1.amazonaws.com",
                    "OriginPath": "/redacted"
                },
                {
                    "Id": "redacted-2",
                    "DomainName": "redacted2.s3.amazonaws.com",
                    "OriginPath": ""
                }
            ]
        }
    }    
}

What I'm trying to do

I want to output the entire document, but:

  • I want to set the ETag value to ""
  • On the 2nd item in the items array, I want to set the OriginPath to a value of my choosing

(this is to support our CI/CD process being able to point the CloudFront distribution to a new folder of just-deployed code within an S3 bucket. I want to modify the existing configuration in those specific ways but leave the rest in-tact.)

Something that works...sort of

A jq filter of . | (.DistributionConfig.Origins.Items[1].OriginPath = "Hello") | .ETag = "" does what I need it to do, resulting in:

{
  "ETag": "", // correctly updated
  "DistributionConfig": {
    "Origins": {
      "Quantity": 2, // correctly retained
      "Items": [
        {
          "Id": "redacted-1",
          "DomainName": "redacted1.us-east-1.amazonaws.com",
          "OriginPath": "/redacted"
        },
        {
          "Id": "redacted-2",
          "DomainName": "redacted2.s3.amazonaws.com",
          "OriginPath": "Hello" // correctly updated
        }
      ]
    }
  }
}

Where I'm having trouble

The solution above works...as long as I'm referring to the 2nd item in the array. But I'm not always sure it will be the second item in the array.

So instead, I'd like to match based on the Id property.

Any solutions I've been able to find that do this seem to filter the document to that section of JSON, rather than matching and updating the value as part of outputting the document.

Question

How can I filter an array for a field that has a certain value, and still output the whole document?

Or put another way -- given the document above, how can I change:

. | (.DistributionConfig.Origins.Items[1].OriginPath = "Hello") | .ETag = ""

To something that lets me refer to .Id="redaacted-2" rather than Items[1]?

A link to a demo

Demo of the script at https://jqplay.org/s/ZQ7XcM5-BY in case anyone wants to try an answer.

peak
  • 105,803
  • 17
  • 152
  • 177
SeanKilleen
  • 8,809
  • 17
  • 80
  • 133

2 Answers2

7

The basic way to accomplish edits such as these is using the syntax:

PATHSPEC = VALUE

or if the value depends on PATHSPEC in a certain way:

PATHSPEC |= VALUE

where PATHSPEC is a jq path specification.

Using this principle, in your case, you could write:

.ETag = ""
| .DistributionConfig.Origins.Items[1].OriginPath = "myvalue"

or, if you want to base the second update on .Id:

.ETag = ""
| .DistributionConfig.Origins.Items[] |=
   if .Id == "redacted-2" then .OriginPath = "myvalue" else . end

The example immediately above can be seen in action at https://jqplay.org/s/u5xbhbSs4l

Variations

There are of course numerous variations. For example, you could use .Quantity as the index of the item to be updated:

.ETag = ""
| .DistributionConfig.Origins.Quantity as $ix
| .DistributionConfig.Origins.Items[$ix - 1].OriginPath = "myvalue"

or more DRYly:

.ETag = ""
| .DistributionConfig.Origins |=
    (.Items[.Quantity  - 1].OriginPath = "myvalue")

peak
  • 105,803
  • 17
  • 152
  • 177
  • I'm not sure I was clear. My concern is that rather than `Items[1]`, I need to effectively say "Items where the Id field is `redacted-2`. Using the array notation currently works, but I cannot guarantee the item I need will be in that position, so I need to go based on the ID field. – SeanKilleen Jun 21 '21 at 20:39
  • Thank you for digging in on this! I am unable to produce a result in my example using your answers. I added a link to a jqplay demo. My goal is to achieve exactly what that example does, but using the Id field instead of Items[1]. – SeanKilleen Jun 21 '21 at 20:53
  • Please see the link to jqplay that I added just now. – peak Jun 21 '21 at 20:59
  • Thank you! @chepner's answer ended up being exactly spot on for what I was looking for, but I've given you an up-vote and appreciate the depth of your answer! – SeanKilleen Jun 21 '21 at 21:09
5

Just a minor variation on @peak's answer, using select to choose which item(s) to update:

.ETag = ""
| (.DistributionConfig.Origins.Items[] | 
      select(.Id == "redacted-2")
  ).OriginPath = "foo"

The second assignment receives the entire original input as its input, so produces it as its output. The parenthesized filter, though, selects only the appropriate element(s) of Items whose OriginPath key you want to update.

peak
  • 105,803
  • 17
  • 152
  • 177
chepner
  • 497,756
  • 71
  • 530
  • 681
  • This ended up being the exact solution I was looking for; the issue I was having is that I was not appropriately understanding the nature of the parenthesized filtering. – SeanKilleen Jun 21 '21 at 21:04