jq, when to use the dot and when not

Question

Consider the following json string:

{
  "data": {
    "search": {
      "repositoryCount": 24,
      "edges": [
        {
          "node": {
            "name": "leumi-leumicard-bank-data-scraper",
            "url": "https://github.com/Urigo/leumi-leumicard-bank-data-scraper",
            "description": "Open bank data for Leumi bank and Leumi card credit card",
            . . . 
        },
        {
          "node": {
            "name": "puppeteer-demo",
            "url": "https://github.com/xJkit/puppeteer-demo",
            "description": "A demo for website scrapping by my puppet :>",
            . . .

If to use jq to select the data, then it needs a dot (.) before it. I.e.:

jq 'data' 
jq: error: data/0 is not defined at <top-level>, line 1:
data
jq: 1 compile error

However jq '.data' works fine, and the selected data becomes:

{
  "search": {
    "repositoryCount": 24,
    "edges": [
      {
      ...

If to use jq to select the search, after the pipe, then it does not need a dot (.) before it. I.e.:

$ jq '.data | {.search} ' 
jq: error: syntax error, unexpected FIELD (Unix shell quoting issues?) at <top-level>, line 1:
.data | {.search}          
jq: 1 compile error

However jq '.data.search' works fine.

Moreover, a more complicated example,

jq '.data.search.edges[] | {node} '

works fine, but

jq '.data.search.edges[] | {node.name} '

gives:

jq: error: syntax error, unexpected FIELD, expecting '}' (Unix shell quoting issues?) at <top-level>, line 1:
.data.search.edges[] | {node.name}                             
jq: 1 compile error

So, all in all, I'm rather confused when to use the dot (.) and when not to, when using jq. Please help. Thx.

This Q/A helped me find a [better Github search approach](https://medium.com/@suntong001/query-github-graphql-9a4547d33bad), FYI. — xpt, Mar 31 '18 at 20:06

score 8 · Accepted Answer · answered Mar 21 '18 at 06:23

8

Perhaps things will be clearer if you begin by thinking of the full data pipeline involved, and recognize when an expression is just an abbreviated form.

In constructing an unabbreviated pipeline, the basic principles regarding "dots" are quite simple:

. refers to the input
.foo is for accessing the value of the key "foo"
.[] is for expanding an array or object

There are many allowed abbreviations. The two that seem to have confused you the most are:

.foo.bar for .foo | .bar
{foo} for {"foo": .foo}

Another important abbreviation is:

E[] for E | .[] where E is a suitably compact expression

With these principles and examples in mind, you should be able to master the details as explained in the jq documentation.

answered Mar 21 '18 at 06:23

peak

105,803
17
152
177

That's wonderful peak, thanks. I tried to use the above principle to make `'.data.search.edges[] | {node.name} '` working for the given example in OP, but I'm still failing. How to make it works, and why the new way is working, not `'.data.search.edges[] | {node.name} '`? Thx. – xpt Mar 21 '18 at 12:18
You cannot get to `{node.name}` by any route because JSON objects have keys and values. The most general single-key JSON object constructor in jq takes the form: `{(KEY): VALUE}`, where KEY and VALUE are appropriate jq expressions. – peak Mar 21 '18 at 12:25
That's strange, because I saw in https://stedolan.github.io/jq/tutorial/#result4 that `'.[] | {message: .commit.message, name: .commit.committer.name}'` is working. I.e., I don't see it any different than my using `{node.name}`. I've also tried `'.data.search.edges[1] | {name: node.name} '`, which seems to be equivalent with `{message: .commit.message}` but that is not working either: `jq: error: node/0 is not defined`. – xpt Mar 21 '18 at 13:04
Ops, `{name: .node.name}` works, let me try the rest of things that I need to try then accept your answer. Mean while, IMO, you may reconsider the `You cannot get to {node.name} by any route` statement, and add/explain *that* case in the answer. Thx. – xpt Mar 21 '18 at 13:08

jq, when to use the dot and when not

1 Answers1