Parse or view JSON data fields using JQ tool utility where field names have a "-" dash in the key name

Question

I have a JSON data file (as shown below) and I'm trying to find field values using jq utility.

It's working fine except for fields if the key name contains a - dash character in it.

How can I get the values of "field-2", "field-three" or "field-three.url" for element under content.book1 (using jq at least)?

I tried the following to get the values but it's giving me the following errors for fields whose key name contains a dash - in it's name. I tried to back slash - character but that didn't help either.

Error types found:

jq: error (at <stdin>:27): null (null) and number (2) cannot be subtracted
jq: 1 compile error

jq: error: three/0 is not defined at <top-level>

jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>

jq: error: syntax error, unexpected INVALID_CHARACTER, expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:

Commands:

$ cat /tmp/my.data.json
{
  "pages": {
    "book1": [
      "page1",
      "page2-para1",
      "page3-para1-sentence1",
      "page3-para2-sentence3-word4"
    ]
  },
  "content": {
    "book1": {
      "name": "giga",
      "url": "-",
      "field1": "value1",
      "field-2": "value-2",
      "field-three": {
        "name": "THIRD",
        "url": "book1/field-three/",
        "short-url": "book1/field-three/chota-chetan"
      },
      "authur": {
        "name": "lori CHUCK",
        "displayIndex": 4
      },
      "route": "/in-gc/hindi-chini-bhai-bhai"
    }
  }
}

$ cat /tmp/my.data.json| jq ".pages"
{
  "book1": [
    "page1",
    "page2-para1",
    "page3-para1-sentence1",
    "page3-para2-sentence3-word4"
  ]
}

$ cat /tmp/my.data.json| jq ".pages.book1[0]"
"page1"

$ cat /tmp/my.data.json| jq ".pages.book1[1]"
"page2-para1"

$ cat /tmp/my.data.json| jq ".content"
{
  "book1": {
    "name": "giga",
    "url": "-",
    "field1": "value1",
    "field-2": "value-2",
    "field-three": {
      "name": "THIRD",
      "url": "book1/field-three/"
    },
    "authur": {
      "name": "lori CHUCK",
      "displayIndex": 4
    },
    "route": "/in-gc/hindi-chini-bhai-bhai"
  }
}

$ cat /tmp/my.data.json| jq ".content.book1"
{
  "name": "giga",
  "url": "-",
  "field1": "value1",
  "field-2": "value-2",
  "field-three": {
    "name": "THIRD",
    "url": "book1/field-three/"
  },
  "authur": {
    "name": "lori CHUCK",
    "displayIndex": 4
  },
  "route": "/in-gc/hindi-chini-bhai-bhai"
}

$ cat /tmp/my.data.json| jq ".content.book1.name"
"giga"

$ cat /tmp/my.data.json| jq ".content.book1.field1"
"value1"

$ cat /tmp/my.data.json| jq ".content.book1.field-2"
jq: error (at <stdin>:27): null (null) and number (2) cannot be subtracted

$ cat /tmp/my.data.json| jq ".content.book1.field-three"
jq: error: three/0 is not defined at <top-level>, line 1:
.content.book1.field-three
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.field-three.url"
jq: error: three/0 is not defined at <top-level>, line 1:
.content.book1.field-three.url
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.field\-2"       
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.content.book1.field\-2                    
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.field\\-2"
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.content.book1.field\-2                    
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.'field-2'"
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.content.book1.'field-2'               
jq: 1 compile error

$ cat /tmp/my.data.json| jq ".content.book1.authur"
{
  "name": "lori CHUCK",
  "displayIndex": 4
}

$ cat /tmp/my.data.json| jq ".content.book1.route"
"/in-gc/hindi-chini-bhai-bhai"

$

PS: I already know egrep so that's not what I'm looking for.

cat /tmp/my.data.json| jq ".content.book1"|egrep "short-url|field-2"
  "field-2": "value-2",
    "short-url": "book1/field-three/chota-chetan"

and someone really did a great job here: https://jqplay.org/

Opened an issue if it's valid: https://github.com/stedolan/jq/issues/1464 — AKS, Aug 16 '17 at 00:37
This question has been asked before, e.g. at https://stackoverflow.com/questions/37344329 — peak, Aug 16 '17 at 01:11
Possible duplicate of [jq not working on tag name with dashes](https://stackoverflow.com/questions/37344329/jq-not-working-on-tag-name-with-dashes) — peak, Aug 16 '17 at 01:11
Agree, In my post the only extra difference that I can see is that, I have listed the type of error messages to catch the post. Thanks peak. — AKS, Aug 16 '17 at 17:22

score 6 · Accepted Answer · answered Aug 16 '17 at 00:40

"-" is used for negation in jq. For key names with special characters such as "-", one cannot use the simplified ".keyname" syntax. There are several alternatives, but the most robust is simply to use the form .["KEY NAME"], which can be abbreviated to ["KEY NAME"] when chained, e.g. .a["b-c"] is shorthand for .a | .["b-c"].

If in doubt, use the pipe explicitly.

For further information, please consult the jq manual and/or https://github.com/stedolan/jq/wiki/FAQ

Nice, I'll accept this as an answer but thanks to every one. — AKS, Aug 16 '17 at 00:48

score 2 · Answer 2 · answered Aug 16 '17 at 00:39

As explained in the jq manual, to handle keys with non-identifier characters like - you can use double quotes.

From the shell this is easiest if you use single quotes around your filter. For example, try the following commands:

cat /tmp/my.data.json | jq '.pages'
cat /tmp/my.data.json | jq '.pages.book1[0]'
cat /tmp/my.data.json | jq '.pages.book1[1]'
cat /tmp/my.data.json | jq '.content'
cat /tmp/my.data.json | jq '.content.book1'
cat /tmp/my.data.json | jq '.content.book1.name'
cat /tmp/my.data.json | jq '.content.book1.field1'
cat /tmp/my.data.json | jq '.content.book1."field-2"'
cat /tmp/my.data.json | jq '.content.book1."field-three"'
cat /tmp/my.data.json | jq '.content.book1."field-three".url'
cat /tmp/my.data.json | jq '.content.book1.authur'
cat /tmp/my.data.json | jq '.content.book1.route'

score 0 · Answer 3 · answered Aug 16 '17 at 00:20

0

I don't know about jq, but you put python in the tags so:

$ cat test.json | python -c "import sys, json; print(json.load(sys.stdin)['content']['book1']['field-three']['name'])"
THIRD

or without the pipe:

$ python -c "import json; print(json.load(open('test.json'))['content']['book1']['field-three']['name'])"

answered Aug 16 '17 at 00:20

Alex Hall

34,833
5
57
89

Thanks Alex; with Python, I already used the same solution like you said but wondering what's bad in `jq` :) and why it's giving me an error for such a simple operation – AKS Aug 16 '17 at 00:23

AKS · Answer 4 · 2017-08-16T17:18:24.703

Hm..took some time but finally it seems like we need to DOUBLE QUOTE it and back slash just the double quote for any key name containing a - in it's name.

$ cat /tmp/my.data.json| jq ".content.book1.\"field-2\""                    
"value-2"

$ cat /tmp/my.data.json| jq ".content.book1.\"field-three\".\"url\""  
"book1/field-three/"

OR if you wrap everything in a single quotes ', then we do NOT need to backslash " double quotes but use double quotes for key names with - in their name.

$ cat /tmp/my.data.json| jq '.content.book1."field-three"."url"'  
"book1/field-three/"

Hope it helps! Took some help/hint from https://jqplay.org/

See this for more: https://github.com/stedolan/jq/issues/38#issuecomment-9770240

Parse or view JSON data fields using JQ tool utility where field names have a "-" dash in the key name

Error types found:

4 Answers4

Linked

Related