Merge sub-hashes within an array in JMESPATH

Question

I have a hash of hashes 5 layers deep. At level 3 I have Docker images. I used JMESPATH query projects.*.docker_images to retrieve these images. Now I have a result like this:

[
  {
    "image1": {
      "registry_url": "blah"
    }
  },
  {
    "image2": {
      "registry_url": "blah"
    },
    "image3": {
      "registry_url": "blah"
    }
  }
]

How do I flatten it to:

{
  "image1": {
    "registry_url": "blah"
  },
  "image2": {
    "registry_url": "blah"
  },
  "image3": {
    "registry_url": "blah"
  }
}

EDIT/CLARIFICATION I'm looking for a generalized solution here, because there will be a variable number of images scattered across a variable number of hashes.

ADDENDUM 1 Using the filtering operator is getting me closer, but not there yet. projects.*.docker_images | [0] creates:

{
  "image1": {
    "registry_url": "blah"
  },
  "image2": {
    "registry_url": "blah"
  }
}

(note image3 is missing)

I don't know if there is problem with your first JSON but if you look well you can see that it miss an curly bracket after the image2, just here: { "image2": { "registry_url": "blah" }}, and after the comma add a curly bracket open and get this at the end: [ { "image1":{ "registry_url":"blah" } }, { "image2":{ "registry_url":"blah" } }, { "image3":{ "registry_url":"blah" } } ] — bosskay972, Jul 17 '19 at 09:02
I just ran the it through jq to be sure, and it parses. That curly bracket you pointed out is the kicker to this problem. Specifically: the first array item has one hash, and the second array item has two hashes. I need all three of them. — EdwardTeach, Jul 19 '19 at 13:44

bosskay972 · Answer 1 · 2019-07-18T12:28:33.677

EDIT: To answer correctly I will consider your JSON in the post as good, so here is my answer:

@.{image1: @[0].image1, image2: @[1].image2, image3: @[1].image3} | [@.image1, @.image2, @.image3] | @.[{image1: @[0]},{image2: @[1]},{image3: @[2]}] with this

you get this:

[  
   {  
      "image1":{  
         "registry_url":"blah"
      }
   },
   {  
      "image2":{  
         "registry_url":"blah"
      }
   },
   {  
      "image3":{  
         "registry_url":"blah"
      }
   }
]

And with this request:

@[*].{name: keys(@)[0], registry_url: join('',*.registry_url)}

you will have this result:

[
  {
    "name": "image1",
    "registry_url": "blah"
  },
  {
    "name": "image2",
    "registry_url": "blah"
  },
  {
    "name": "image3",
    "registry_url": "blah"
  }
]

If there is any problem with registry_url(with the type) you od not have to use join function you can use max or min function, just I was need a function to convert an array to a value :) !

If you want to flatten the result again (withouth square bracket) you normally can't do it in an easy way(I don't think it's possible, but if it's possible it would be a little bit tricky).

so the entire request is: @.{image1: @[0].image1, image2: @[1].image2, image3: @[1].image3} | [@.image1, @.image2, @.image3] | @.[{image1: @[0]},{image2: @[1]},{image3: @[2]}] | @[*].{name: keys(@)[0], registry_url: join('',*.registry_url)}

It's work in fact, but I don't think it's the best way !

And if you want without square bracket(only curly bracket) do this(but if we do this we have to put attribute on the object:

so here is the request: @.{image1: @[0].image1, image2: @[1].image2, image3: @[1].image3} | [@.image1, @.image2, @.image3] | @.[{image1: @[0]},{image2: @[1]},{image3: @[2]}] | @[*].{name: keys(@)[0], registry_url: join('',*.registry_url)} | @.{obj1: @[0], obj2: @[1], obj3: @[2]}}

Here's the result:

{
  "obj1": {
    "name": "image1",
    "registry_url": "blah"
  },
  "obj2": {
    "name": "image2",
    "registry_url": "blah"
  },
  "obj3": {
    "name": "image3",
    "registry_url": "blah"
  }
}

Yup, my json above is indeed weird and that's why I ran into trouble :( — EdwardTeach, Jul 18 '19 at 01:00
The JSON in your first post was the REAL JSON ?? There was no error in it ? Oh okay I will try to edit my answer to answer to your post correctly — bosskay972, Jul 18 '19 at 10:11
I removed the real registry URLs, but other than that, yes. The structure of it is what is weird and giving me trouble. — EdwardTeach, Jul 18 '19 at 15:48
But the url is a string(if yes, it will work anyway). And, finally my post has it answered to your question ? — bosskay972, Jul 19 '19 at 09:18
Hi, thanks for your answer. This looks like it would work for specifically the json I posted above, however I am looking for a generalized solution. Thanks for helping me improve my question! I updated it to add a note that I'm looking for a generalized solution. — EdwardTeach, Jul 19 '19 at 13:56

score 0 · Answer 2 · answered May 17 '19 at 19:36

I have a solution, but not an answer. I will leave my solution up here, but leave the question open in case somebody arrives at a better answer.

The solution is: restructure the source data from hashes/dicts/objects into arrays/lists, like so:

[
  [
    {
      "name": "image1",
      "registry_url": "blah"
    }
  ],
  [
    {
      "name": "image2",
      "registry_url": "blah"
    },
    {
      "name": "image3",
      "registry_url": "blah"
    }
  ]
]

Then retrieve the data with projects.*.docker_images[]. Now the flatten [] works as expected, and gives me an array of objects.

Merge sub-hashes within an array in JMESPATH

2 Answers2