0

tl;dr

I'm trying to get a JSON representation of juju status output which objectifies the machine, the instance-id, and the unit such that the JSON looks similar to:

{
  "0": {
          "instance-id": "i-xxxxxxxxxxxxxx",
          "unit-name": "easyrsa/0"
       },
  "1": {
          "instance-id": "i-xxxxxxxxxxxxxx",
          "unit-name": "etcd/0"
       },
  "2": {
          "instance-id": "i-xxxxxxxxxxxxxx",
          "unit-name": "kubeapi-load-balancer/0"
       },
  "10": {
          "instance-id": "i-xxxxxxxxxxxxxx",
          "unit-name": "kubernetes-worker/4"
       },
  "11": {
          "instance-id": "i-xxxxxxxxxxxxxx",
          "unit-name": "kubernetes-worker/5"
       },
  "12": {
          "instance-id": "i-xxxxxxxxxxxxxx",
          "unit-name": "kubernetes-master/3"
       }
}

The long of it:

Munging juju status --format json is complicated for me because the data to be munged is split between two different main objects in the JSON. Since the keys for the machines are not iterable, yet, I'm not able to reference them later like I could an array -- or at least that's where I think I'm getting messed up.

Some things I've tried but failed (mostly because I struggle with how to store the .machines keys for use later)...

juju status --format json | jq -r '.machines as $m | .machines | [foreach keys[] as $item ({m: $item, id: $m[$item]."instance-id"})]'

... | juju status --format json | jq -r '.machines | keys[] as $k...

... | juju status --format json | jq -r '.machines | keys[] as $k |...

Either I don't get the result I need or I get a syntax error. I've never had a need to use foreach in jq context. In fact, this is the most complicated thing I've tried to accomplish with jq so I'm way outside my comfort zone here. Any assistance would be greatly appreciated.

Here's an sample juju status JSON object:

{
  "model": {
    "name": "xxxxxxxxxx",
    "controller": "xxxxxxxxxx",
    "cloud": "xxxxxxxxxx",
    "region": "xxxxxxxxxx",
    "version": "xxxxxxxxxx",
    "model-status": {
      "current": "xxxxxxxxxx",
      "since": "xxxxxxxxxx"
    },
    "sla": "xxxxxxxxxx"
  },
  "machines": {
    "0": {
      "juju-status": {
        "current": "xxxxxxxxxx",
        "since": "xxxxxxxxxx",
        "version": "xxxxxxxxxx"
      },
      "dns-name": "xxxxxxxxxx",
      "ip-addresses": [
        "10.0.0.229",
        "252.0.229.1"
      ],
      "instance-id": "i-xxxxxxxxxxxxxx",
      "machine-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "series": "xxxxxxxxxx",
      "network-interfaces": {
        "eth0": {
          "ip-addresses": [
            "10.0.0.229"
          ],
          "mac-address": "xxxxxxxxxx",
          "gateway": "xxxxxxxxxx",
          "is-up": true
        },
        "fan-252": {
          "ip-addresses": [
            "252.0.229.1"
          ],
          "mac-address": "xxxxxxxxxx",
          "is-up": true
        }
      },
      "constraints": "xxxxxxxxxx",
      "hardware": "xxxxxxxxxx"
    },
    "1": {
      "juju-status": {
        "current": "xxxxxxxxxx",
        "since": "xxxxxxxxxx",
        "version": "xxxxxxxxxx"
      },
      "dns-name": "xxxxxxxxxx",
      "ip-addresses": [
        "10.0.0.61",
        "252.0.61.1"
      ],
      "instance-id": "i-xxxxxxxxxxxxxx",
      "machine-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "series": "xxxxxxxxxx",
      "network-interfaces": {
        "eth0": {
          "ip-addresses": [
            "10.0.0.61"
          ],
          "mac-address": "xxxxxxxxxx",
          "gateway": "xxxxxxxxxx",
          "is-up": true
        },
        "fan-252": {
          "ip-addresses": [
            "252.0.61.1"
          ],
          "mac-address": "xxxxxxxxxx",
          "is-up": true
        }
      },
      "constraints": "xxxxxxxxxx",
      "hardware": "xxxxxxxxxx"
    },
    "10": {
      "juju-status": {
        "current": "xxxxxxxxxx",
        "since": "xxxxxxxxxx",
        "version": "xxxxxxxxxx"
      },
      "dns-name": "xxxxxxxxxx",
      "ip-addresses": [
        "10.0.0.37",
        "252.0.37.1"
      ],
      "instance-id": "i-xxxxxxxxxxxxxx",
      "machine-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "series": "xxxxxxxxxx",
      "network-interfaces": {
        "ens5": {
          "ip-addresses": [
            "10.0.0.37"
          ],
          "mac-address": "xxxxxxxxxx",
          "gateway": "xxxxxxxxxx",
          "is-up": true
        },
        "fan-252": {
          "ip-addresses": [
            "252.0.37.1"
          ],
          "mac-address": "xxxxxxxxxx",
          "is-up": true
        }
      },
      "constraints": "xxxxxxxxxx",
      "hardware": "xxxxxxxxxx"
    },
    "11": {
      "juju-status": {
        "current": "xxxxxxxxxx",
        "since": "xxxxxxxxxx",
        "version": "xxxxxxxxxx"
      },
      "dns-name": "xxxxxxxxxx",
      "ip-addresses": [
        "10.0.0.54"
      ],
      "instance-id": "i-xxxxxxxxxxxxxx",
      "machine-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "series": "xxxxxxxxxx",
      "network-interfaces": {
        "ens5": {
          "ip-addresses": [
            "10.0.0.54"
          ],
          "mac-address": "xxxxxxxxxx",
          "gateway": "xxxxxxxxxx",
          "is-up": true
        }
      },
      "constraints": "xxxxxxxxxx",
      "hardware": "xxxxxxxxxx"
    },
    "12": {
      "juju-status": {
        "current": "xxxxxxxxxx",
        "since": "xxxxxxxxxx",
        "version": "xxxxxxxxxx"
      },
      "dns-name": "xxxxxxxxxx",
      "ip-addresses": [
        "10.0.0.101"
      ],
      "instance-id": "i-xxxxxxxxxxxxxx",
      "machine-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "series": "xxxxxxxxxx",
      "network-interfaces": {
        "ens5": {
          "ip-addresses": [
            "10.0.0.101"
          ],
          "mac-address": "xxxxxxxxxx",
          "gateway": "xxxxxxxxxx",
          "is-up": true
        }
      },
      "constraints": "xxxxxxxxxx",
      "hardware": "xxxxxxxxxx"
    },
    "2": {
      "juju-status": {
        "current": "xxxxxxxxxx",
        "since": "xxxxxxxxxx",
        "version": "xxxxxxxxxx"
      },
      "dns-name": "xxxxxxxxxx",
      "ip-addresses": [
        "10.0.0.184",
        "252.0.184.1"
      ],
      "instance-id": "i-xxxxxxxxxxxxxx",
      "machine-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "series": "xxxxxxxxxx",
      "network-interfaces": {
        "eth0": {
          "ip-addresses": [
            "10.0.0.184"
          ],
          "mac-address": "xxxxxxxxxx",
          "gateway": "xxxxxxxxxx",
          "is-up": true
        },
        "fan-252": {
          "ip-addresses": [
            "252.0.184.1"
          ],
          "mac-address": "xxxxxxxxxx",
          "is-up": true
        }
      },
      "constraints": "xxxxxxxxxx",
      "hardware": "xxxxxxxxxx"
    }
  },
  "applications": {
    "easyrsa": {
      "charm": "xxxxxxxxxx",
      "series": "xxxxxxxxxx",
      "os": "xxxxxxxxxx",
      "charm-origin": "xxxxxxxxxx",
      "charm-name": "xxxxxxxxxx",
      "charm-rev": 39,
      "can-upgrade-to": "xxxxxxxxxx",
      "exposed": false,
      "application-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "relations": {
        "client": [
          "etcd",
          "kubeapi-load-balancer",
          "kubernetes-master",
          "kubernetes-worker"
        ]
      },
      "units": {
        "easyrsa/0": {
          "workload-status": {
            "current": "xxxxxxxxxx",
            "message": "xxxxxxxxxx",
            "since": "xxxxxxxxxx"
          },
          "juju-status": {
            "current": "xxxxxxxxxx",
            "since": "xxxxxxxxxx",
            "version": "xxxxxxxxxx"
          },
          "leader": true,
          "machine": "0",
          "public-address": "xxxxxxxxxx"
        }
      },
      "version": "xxxxxxxxxx"
    },
    "etcd": {
      "charm": "xxxxxxxxxx",
      "series": "xxxxxxxxxx",
      "os": "xxxxxxxxxx",
      "charm-origin": "xxxxxxxxxx",
      "charm-name": "xxxxxxxxxx",
      "charm-rev": 77,
      "can-upgrade-to": "xxxxxxxxxx",
      "exposed": false,
      "application-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "relations": {
        "certificates": [
          "easyrsa"
        ],
        "cluster": [
          "etcd"
        ],
        "db": [
          "flannel",
          "kubernetes-master"
        ]
      },
      "units": {
        "etcd/0": {
          "workload-status": {
            "current": "xxxxxxxxxx",
            "message": "xxxxxxxxxx",
            "since": "xxxxxxxxxx"
          },
          "juju-status": {
            "current": "xxxxxxxxxx",
            "since": "xxxxxxxxxx",
            "version": "xxxxxxxxxx"
          },
          "leader": true,
          "machine": "1",
          "open-ports": [
            "2379/tcp"
          ],
          "public-address": "xxxxxxxxxx"
        }
      },
      "version": "xxxxxxxxxx"
    },
    "flannel": {
      "charm": "xxxxxxxxxx",
      "series": "xxxxxxxxxx",
      "os": "xxxxxxxxxx",
      "charm-origin": "xxxxxxxxxx",
      "charm-name": "xxxxxxxxxx",
      "charm-rev": 52,
      "can-upgrade-to": "xxxxxxxxxx",
      "exposed": false,
      "application-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "relations": {
        "cni": [
          "kubernetes-master",
          "kubernetes-worker"
        ],
        "etcd": [
          "etcd"
        ]
      },
      "subordinate-to": [
        "kubernetes-master",
        "kubernetes-worker"
      ],
      "version": "xxxxxxxxxx"
    },
    "kubeapi-load-balancer": {
      "charm": "xxxxxxxxxx",
      "series": "xxxxxxxxxx",
      "os": "xxxxxxxxxx",
      "charm-origin": "xxxxxxxxxx",
      "charm-name": "xxxxxxxxxx",
      "charm-rev": 57,
      "can-upgrade-to": "xxxxxxxxxx",
      "exposed": true,
      "application-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "relations": {
        "apiserver": [
          "kubernetes-master"
        ],
        "certificates": [
          "easyrsa"
        ],
        "loadbalancer": [
          "kubernetes-master"
        ],
        "website": [
          "kubernetes-worker"
        ]
      },
      "units": {
        "kubeapi-load-balancer/0": {
          "workload-status": {
            "current": "xxxxxxxxxx",
            "message": "xxxxxxxxxx",
            "since": "xxxxxxxxxx"
          },
          "juju-status": {
            "current": "xxxxxxxxxx",
            "since": "xxxxxxxxxx",
            "version": "xxxxxxxxxx"
          },
          "leader": true,
          "machine": "2",
          "open-ports": [
            "443/tcp"
          ],
          "public-address": "xxxxxxxxxx"
        }
      },
      "version": "xxxxxxxxxx"
    },
    "kubernetes-master": {
      "charm": "xxxxxxxxxx",
      "series": "xxxxxxxxxx",
      "os": "xxxxxxxxxx",
      "charm-origin": "xxxxxxxxxx",
      "charm-name": "xxxxxxxxxx",
      "charm-rev": 102,
      "can-upgrade-to": "xxxxxxxxxx",
      "exposed": false,
      "application-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "relations": {
        "certificates": [
          "easyrsa"
        ],
        "cni": [
          "flannel"
        ],
        "etcd": [
          "etcd"
        ],
        "kube-api-endpoint": [
          "kubeapi-load-balancer"
        ],
        "kube-control": [
          "kubernetes-worker"
        ],
        "loadbalancer": [
          "kubeapi-load-balancer"
        ]
      },
      "units": {
        "kubernetes-master/3": {
          "workload-status": {
            "current": "xxxxxxxxxx",
            "message": "xxxxxxxxxx",
            "since": "xxxxxxxxxx"
          },
          "juju-status": {
            "current": "xxxxxxxxxx",
            "since": "xxxxxxxxxx",
            "version": "xxxxxxxxxx"
          },
          "leader": true,
          "machine": "12",
          "open-ports": [
            "6443/tcp"
          ],
          "public-address": "xxxxxxxxxx",
          "subordinates": {
            "flannel/9": {
              "workload-status": {
                "current": "xxxxxxxxxx",
                "message": "xxxxxxxxxx",
                "since": "xxxxxxxxxx"
              },
              "juju-status": {
                "current": "xxxxxxxxxx",
                "since": "xxxxxxxxxx",
                "version": "xxxxxxxxxx"
              },
              "upgrading-from": "xxxxxxxxxx",
              "public-address": "xxxxxxxxxx"
            }
          }
        }
      },
      "version": "xxxxxxxxxx"
    },
    "kubernetes-worker": {
      "charm": "xxxxxxxxxx",
      "series": "xxxxxxxxxx",
      "os": "xxxxxxxxxx",
      "charm-origin": "xxxxxxxxxx",
      "charm-name": "xxxxxxxxxx",
      "charm-rev": 114,
      "can-upgrade-to": "xxxxxxxxxx",
      "exposed": false,
      "application-status": {
        "current": "xxxxxxxxxx",
        "message": "xxxxxxxxxx",
        "since": "xxxxxxxxxx"
      },
      "relations": {
        "certificates": [
          "easyrsa"
        ],
        "cni": [
          "flannel"
        ],
        "kube-api-endpoint": [
          "kubeapi-load-balancer"
        ],
        "kube-control": [
          "kubernetes-master"
        ]
      },
      "units": {
        "kubernetes-worker/4": {
          "workload-status": {
            "current": "xxxxxxxxxx",
            "message": "xxxxxxxxxx",
            "since": "xxxxxxxxxx"
          },
          "juju-status": {
            "current": "xxxxxxxxxx",
            "since": "xxxxxxxxxx",
            "version": "xxxxxxxxxx"
          },
          "machine": "10",
          "open-ports": [
            "80/tcp",
            "443/tcp"
          ],
          "public-address": "xxxxxxxxxx",
          "subordinates": {
            "flannel/7": {
              "workload-status": {
                "current": "xxxxxxxxxx",
                "message": "xxxxxxxxxx",
                "since": "xxxxxxxxxx"
              },
              "juju-status": {
                "current": "xxxxxxxxxx",
                "since": "xxxxxxxxxx",
                "version": "xxxxxxxxxx"
              },
              "upgrading-from": "xxxxxxxxxx",
              "public-address": "xxxxxxxxxx"
            }
          }
        },
        "kubernetes-worker/5": {
          "workload-status": {
            "current": "xxxxxxxxxx",
            "message": "xxxxxxxxxx",
            "since": "xxxxxxxxxx"
          },
          "juju-status": {
            "current": "xxxxxxxxxx",
            "since": "xxxxxxxxxx",
            "version": "xxxxxxxxxx"
          },
          "leader": true,
          "machine": "11",
          "open-ports": [
            "80/tcp",
            "443/tcp"
          ],
          "public-address": "xxxxxxxxxx",
          "subordinates": {
            "flannel/8": {
              "workload-status": {
                "current": "xxxxxxxxxx",
                "message": "xxxxxxxxxx",
                "since": "xxxxxxxxxx"
              },
              "juju-status": {
                "current": "xxxxxxxxxx",
                "since": "xxxxxxxxxx",
                "version": "xxxxxxxxxx"
              },
              "leader": true,
              "upgrading-from": "xxxxxxxxxx",
              "public-address": "xxxxxxxxxx"
            }
          }
        }
      },
      "version": "xxxxxxxxxx"
    }
  }
}
Jim
  • 1,499
  • 1
  • 24
  • 43
  • Thanks @peak I cleaned it up and fixed the end result to be complete. Do let me know if anything else needs polishing. – Jim Apr 27 '18 at 18:01
  • Where does "i-xxxxxxxxxxxxxx" come from? Couldn't you arrange to have the values of "instance-id" differ so it's clear where they come from? Also, machine "10" is associated with "kubernetes-worker/4" in the source, but with "etcd/0" in the expected output. It might help everyone if you could make the entire example **minimal**. – peak Apr 28 '18 at 02:08
  • @peak providing minimal input is difficult provided that it's dynamic and I know next to nothing about it iteratively. Removing everything except what I'm seeking could likely provide a falsely represented dataset given future iterations. However, I re-verified the source data, though (made fixes). I'm not worried about the actual data (IE the rvals) as much as I am about getting right object keys I'm seeking from the `.machines` and `.applications` parent objects for the LHS. In other words, if I can get the right keys, the data will be correct. I don't understand jq well enough to do this. – Jim Apr 29 '18 at 09:22

1 Answers1

1

It is unclear to me how exactly the value of "unit-name" is supposed to be derived, but the following should get you on your way:

def machine($id):
  first(.applications[]
    | (.units? // empty)
    | to_entries[]
    | select(.value.machine == $id)
    | .key ) ;

. as $in
| .machines
| to_entries[]
| .key as $key
| {($key): {
             "unit-name": (.value |.["instance-id"]),
             "machine": ($in|machine($key))
           } }

With your input, this produces a stream of objects beginning:

{
  "0": {
    "unit-name": "xxxxxxxxxx",
    "machine": "easyrsa/0"
  }
}
{
  "1": {
    "unit-name": "xxxxxxxxxx",
    "machine": "etcd/0"
  }
}

Takeaways

The solution illustrates three noteworthy points:

  1. to_entries is useful when one has to work with key/value combinations when the specific key (or keys) of interest is (or are) not known beforehand; an alternative is to use keys_unsorted;

  2. dollar-variables are handy when pulling together information from distinct sources;

  3. jq's support for function definitions makes it possible to write programs that are more readable and maintainable than they would otherwise be.

peak
  • 105,803
  • 17
  • 152
  • 177
  • Works like a champ! Now, I never would have been able to conceive this as the solution. I'm going to study this answer and learn. Thank you @peak. – Jim Apr 29 '18 at 18:45