How to concatenate values for the same key in selecting from a json based har file using jq

Question

I am trying to generate a .csv file from processing a .har (JSON based) file.

My current command is like this (note that I need to have ascii_downcase as name is found in upper, lower and Initcaps):


    cat "$har1".har | $jq -r '[ "Name", "StartTime", "Cache", "Encoding", "ECID", "Method", "Size", "Time", "Blocked", "Send", "Wait", "Receive", "Type" ], (.log.entries[] | [
        .request.url,
        .startedDateTime,
        ((.response.headers[] | select(.name | ascii_downcase == "cache-control").value) // ""),
        ((.response.headers[] | select(.name | ascii_downcase == "content-encoding").value) // ""),
        ((.response.headers[] | select(.name | ascii_downcase == "x-oracle-dms-ecid").value) // ""),
            .request.method,
        (if .response.bodySize == 0 then .response.content.size else .response.bodySize end),
        .time,
        .timings.blocked,
        .timings.send,
        .timings.wait,
        .timings.receive,
        ._resourceType ]) | @csv' > "$har1".csv

The har snippet that is causing issues in the response, is multiple entries for "Cache-Control", Like below:


        {
          "name": "Cache-Control",
          "value": "public"
        },
        {
          "name": "Cache-Control",
          "value": "max-age=31536000"
        },

In the following portion of json:

    "response": {
      "status": 200,
      "statusText": "OK",
      "httpVersion": "HTTP/1.1",
      "headers": [
        {
          "name": "Date",
          "value": "Thu, 06 May 2021 04:53:24 GMT"
        },
        {
          "name": "Server",
          "value": "Oracle-HTTP-Server"
        },
        {
          "name": "X-XSS-Protection",
          "value": "1; mode=block"
        },
        {
          "name": "Cache-Control",
          "value": "public"
        },
        {
          "name": "Cache-Control",
          "value": "max-age=31536000"
        },

The values could be anything, so do not want to hard code anything. The current result due to this in .csv is like:

"public","max-age=31536000"

they are split as 2 columns.

"xx.js","2021-05-06T06:14:10.505Z","public","max-age=31536000","gzip"...

I need to get this as:

"public;max-age=31536000"

with ; as separator, but in same column of .csv.

"xx.js","2021-05-06T06:14:10.505Z","public;max-age=31536000","gzip"...

I tried groups_by, group_by examples, but got errors one way or another.

oguz ismail's suggestion had unequal "(" and ")".

I tried using various combinations based on the suggestion, and realize:

It still does not merge the columns as desired with ";" into one column in the generated .csv.
It has added 3 null columns before and 6 after the .name | ascii_downcase == "cache-control" column.

Original code:

((.response.headers[] | select(.name | ascii_downcase == "cache-control").value) // ""),
Original response:
"xx.js","2021-05-06T06:14:10.505Z","public","max-age=31536000","gzip",

Expected response:
"xx.js","2021-05-06T06:14:10.505Z","public;max-age=31536000","gzip",

first attempt:

((.response.headers[] | ([select(.name | ascii_downcase == "cache-control").value]) | join(";")) // ""),
"xx.js","2021-05-06T06:14:10.505Z","","","","public","max-age=31536000","","","","","","","gzip",

second attempt:

((.response.headers[] | [select(.name | ascii_downcase == "cache-control").value] | join(";")) // ""),
"xx.js","2021-05-06T06:14:10.505Z","","","","public","max-age=31536000","","","","","","","gzip"

third attempt:

((.response.headers[] | [(select(.name | ascii_downcase == "cache-control").value)] | join(";")) // ""),
"xx.js","2021-05-06T06:14:10.505Z","","","","public","max-age=31536000","","","","","","","gzip",

I have tried reformatting a bit, but not sure if I got it all right. — Benjamin W., May 13 '21 at 18:28
Thanks Benjamin, I retried re-formatting again, hopefully got it better this time. — Sanjay S, May 13 '21 at 18:33
`([select(.name | ascii_downcase == "cache-control").value)] | join(";")) // ""` should do the trick. — oguz ismail, May 13 '21 at 18:38
This one works: ([.response.headers[] | select(.name | ascii_downcase == "cache-control").value] | join(";")), — Sanjay S, May 14 '21 at 00:49

score 0 · Answer 1 · answered May 14 '21 at 10:28

Your general problem is that .response.headers[] produces multiple results and you have nothing to gather those results into a single value. You can either wrap an array construction around the expression, i.e. [.response.headers[]| <...more filters here> ], or you can use map instead of the [] operator.

Using an array constructor:

([.response.headers[] | select(.name | ascii_downcase == "cache-control").value] | join(";"))

Using map:

(.response.headers | map(select(.name | ascii_downcase == "cache-control").value | join(";"))

Thanks. I used the first construct successfully. – Sanjay S May 15 '21 at 15:01 — Sanjay S, May 15 '21 at 15:01

How to concatenate values for the same key in selecting from a json based har file using jq

1 Answers1