How to iterate through all values of a nested map in groovy

Question

I have a nested map of unknown structure, the goal is to iterate through every value in the map, check the values of a certain condition (e.g. null) and replace those values with something else. I can see how to do it if the map structure is known, but the issue here is that it is unknown.

For example this could be the map structure passed (or could have any number of nested maps):

def map = [
          a:5,
          b:"r",
          c:[ a1:0, b1:null ],
          d:[ a2: [ a3:"", b3:99 ], b2:null ],
          ...
]

Normally for a simple map one would use a this to update values:

map.each { it.value == null ? it.value = "" : "" }

However with an nested map structure this approach will not work.

Is there an efficient way to iterating through all the nested values of an unknown map to investigate and update the values?

score 6 · Answer 1 · answered Jul 24 '18 at 05:22

6

You can run that with each, but you need to recurse for maps then. E.g. see deNull.

def deNull(def root) {
    root.each{
        if (it.value instanceof Map) {
            deNull(it.value)
        } else if (it.value==null) {
            it.value = ""
        }
    }
}

def map = [
    a:5,
    b:"r",
    c:[ a1:0, b1:null ],
    d:[ a2: [ a3:"", b3:99 ], b2:null ],
]
println(map.tap{ deNull it }.inspect())
// => ['a':5, 'b':'r', 'c':['a1':0, 'b1':''], 'd':['a2':['a3':'', 'b3':99], 'b2':'']]

For a proper approach i'd also pass in a closure for "what to do" instead of just dealing with the "de-null-ing" here (which makes that reuseable) and name it postwalkMap or something like that.

answered Jul 24 '18 at 05:22

cfrick

35,203
6
56
68

Nice! FYI you may want to tweak the final execution, LinkedHashMap doesn't support tap() -> `No signature of method: java.util.LinkedHashMap.tap() is applicable for argument types: (Script1$_run_closure1)` – rboy Jul 24 '18 at 15:17
1

`tap` is in Groovy 2.5 and it's not relevant to the question. You can just as well use `def map = ...; deNull map; println(map.inspect())` – cfrick Jul 25 '18 at 03:41

score 3 · Answer 2 · answered Jul 24 '18 at 05:56

You can also use Map.replaceAll(BiFunction<String, Serializable, Serializable>) func) to replace all nulls recursively in a map of an unknown structure.

Consider following example:

import java.util.function.BiFunction

def map = [
        a: 5,
        b: "r",
        c: [a1: 0, b1: null],
        d: [a2: [a3: "", b3: 99], b2: null]
]

def nullsWith(Object replacement) {
    return { String key, Serializable value ->
        if (value instanceof Map) {
            value.replaceAll(nullsWith(replacement))
            return value
        }
        return value == null ? replacement : value
    } as BiFunction<String, Serializable, Serializable>
}

map.replaceAll nullsWith("null replacement")

println map.inspect()

Output:

['a':5, 'b':'r', 'c':['a1':0, 'b1':'null replacement'], 'd':['a2':['a3':'', 'b3':99], 'b2':'null replacement']]

score 2 · Accepted Answer · answered Jul 28 '18 at 05:41

This is an improved version of what @cfrick has posted above.

Key improvements:

it can loop through a nested object which contains nested maps and lists
it can replace a null with a custom value

.

def deNull(root, replaceNullWith = "") {
    if (root instanceof List) {
        root.collect {
            if (it instanceof Map) {
                deNull(it, replaceNullWith)
            } else if (it instanceof List) {
                deNull(it, replaceNullWith)
            } else if (it == null) {
                replaceNullWith
            } else {
                it
            }
        }
    } else if (root instanceof Map) {
        root.each {
            if (it.value instanceof Map) {
                deNull(it.value, replaceNullWith)
            } else if (it.value instanceof List) {
                it.value = deNull(it.value, replaceNullWith)
            } else if (it.value == null) {
                it.value = replaceNullWith
            }
        }
    }
}

def map = [
    a:5,
    b:"r",
    c:[ a1:0, b1:null ],
    d:[ a2: [ a3:"", b3:99 ], b2:null, b3: [2, "x", [], [null], null] ],
]

map = deNull(map)
println map.inspect()

Thanks for your work on this answer! I posted another answer with some tweaks so that this function could be used for more than just `null` values in my specific use case (Jenkins). — Kyle Pittman, Dec 05 '19 at 17:42

score 0 · Answer 4 · answered Dec 05 '19 at 17:41

Building on the post from @rboy - I am working on serializing Jenkins build results as JSON and I keep running into stack overflow errors since the build result has a cyclic reference.

In the build results, there are references to previousBuild and nextBuild. If Job 2 has a previousBuild pointing to Job 1, and Job 1 has a nextBuild pointing to Job 2, then the entire object cannot be serialized since there is a cyclic reference.

To avoid this, I wanted a way that I could remove/replace any instances of the build result classes from my object.

Modifying the other post, I came up with the following:

import org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper

// helpers
isClosure = {it instanceof Closure}
thunkify = {it -> {_ -> it}}
thunkifyValue = {isClosure(it) ? it : thunkify(it)}
makePredicate = {isClosure(it) ? it : it.&equals}
isAnyType = {types, value -> types.any{it.isInstance(value)}}
isMapOrList = isAnyType.curry([List, Map])

def cleanse(root, _predicate, _transform) {
    def predicate = makePredicate(_predicate)
    def transform = thunkifyValue(_transform)

    if (root instanceof List) {
        return root.collect {
            if (isMapOrList(it)) {
                it = cleanse(it, predicate, transform)
            } else if (predicate(it)) {
                it = transform(it)
            }
            return it
        }
    } else if (root instanceof Map) {
        return root.collectEntries {k,v ->
            if (isMapOrList(v)) {
                v = cleanse(v, predicate, transform)
            } else if (predicate(v)) {
                v = transform(v)
            }
            return [(k): v]
        }
    } else {
        return root
    }
}

// basic usage - pass raw values
// jobs is an array of Jenkins job results
// replaces any occurrence of the value null with the value 'replaced null with string'
cleanse(jobs, null, 'replaced null with string')

// advanced usage - pass closures
// We will replace any value that is an instance of RunWrapper
// with the calculated value "Replaced Build Result - $it.inspect()"
cleanse(jobs, {it instanceof RunWrapper}, {"Replaced Build Result - ${it.inspect()}"})

After calling this on my jenkins results I am able to serialize them as JSON without getting a StackOverflowError from the cyclic dependencies. This is how I use the cleanse function in my code:

// testJobs contains all of the results from each `build()` in my pipeline
cleansed = cleanse(testJobs.collect {
    def props = it.properties
    // Something about the formatting of the values in rawBuild will cause StackOverflowError when creating json
    props['rawBuild'] = props['rawBuild'].toString()
    props
}, {it instanceof RunWrapper}, 'Replaced Build Result')
cleansed = cleanse(cleansed, {it instanceof Class}, 'Class stub')
// `cleansed` now has no values that are instances of RunWrapper and no values that are classes
// both of those will cause issues during JSON serialization

// render JSON
cleansedJson = groovy.json.JsonOutput.toJson(cleansed)

How to iterate through all values of a nested map in groovy

4 Answers4

Linked