Compare-Object in Powershell for 2 objects based on a field within. Objects populated by JSON and XML

Question

Apologies for my lack of powershell knowledge, have been searching far and wide for a solution as i am not much of a programmer.

Background:

I am currently trying to standardise some site settings in Incapsula. To do this i want to maintain a local XML with rules and use some powershell to pull down the existing rules and compare them with what is there to ensure im not doubling up. I am taking this approach of trying to only apply the deltas as:

1. For most settings incapsula is not smart enough to know it already exists
2. What can be posted to the API is different varies from what is returned by the API

Examples:

Below is an example of what the API will return on request, this is in a JSON format.

JSON FROM WEBSITE

{
"security": {
        "waf": {
            "rules": [{
                "id": "api.threats.sql_injection",
                "exceptions": [{
                    "values": [{
                        "urls": [{
                            "value": "google.com/thisurl",
                            "pattern": "EQUALS"
                        }],
                        "id": "api.rule_exception_type.url",
                        "name": "URL"
                    }],
                    "id": 256354634
                }]
            }, {
                "id": "api.threats.cross_site_scripting",
                "action": "api.threats.action.block_request",
                "exceptions": [{
                    "values": [{
                        "urls": [{
                            "value": "google.com/anotherurl",
                            "pattern": "EQUALS"
                        }],
                        "id": "api.rule_exception_type.url",
                        "name": "URL"
                    }],
                    "id": 78908790780
                }]
            }]
        }
    }
}

And this is the format of the XML with our specific site settings in it

OUR XML RULES
    <waf>
    <ruleset>
        <rule>
            <id>api.threats.sql_injection</id>
                <exceptions>
                    <exception>
                        <type>api.rule_exception_type.url</type>
                        <url>google.com/thisurl</url>   
                    </exception>
                    <exception>
                        <type>api.rule_exception_type.url</type>
                        <url>google.com/thisanotherurl</url>
                    </exception>
                </exceptions>
        </rule>
        <rule> 
            <id>api.threats.cross_site_scripting</id>
                <exceptions>
                    <exception>
                        <type>api.rule_exception_type.url</type>
                        <url>google.com/anotherurl</url>
                    </exception>
                    <exception>
                        <type>api.rule_exception_type.url</type>
                        <url>google.com/anotherurl2</url>
                    </exception> 
                </exceptions>
        </rule> 
    </ruleset>
</waf>

I have successfully been able to compare other settings from the site against the XML using the compare-object command, however they had a bit simpler nesting and didn't give me as much trouble. I'm stuck to whether it is a logic problem or a limitation with compare object. An example code is below, it will require the supplied json and xml saved as stack.json/xml in the same directory and should produce the mentioned result :

    $existingWaf = Get-Content -Path stack.json | ConvertFrom-Json
    [xml]$xmlFile = Get-Content -Path stack.xml 

    foreach ($rule in $xmlFile)
    {
        $ruleSet = $rule.waf.ruleset
    }

    foreach ($siteRule in $ExistingWaf.security.waf.rules)
        {
            foreach ($xmlRule in $ruleSet)
            {
                if ($xmlRule.rule.id -eq $siteRule.id)
                    {
                        write-output "yes"
                        $delta = Compare-Object -ReferenceObject @($siteRule.exceptions.values.urls.value | Select-Object) -DifferenceObject @($xmlRule.rule.exceptions.exception.url | Select-Object) -IncludeEqual | where {$xmlRule.rule.id -eq $siteRule.id}
                        $delta

                    }
            }
        }

This is kind of working but not quite what i wanted. I do get a compare between the objects but not for the specific id's, it shows me the results below:

    InputObject                                 SideIndicator
    -----------                                 -------------
    google.com/thisurl                               ==
    google.com/thisanotherurl                        =>
    google.com/anotherurl                            =>
    google.com/anotherurl2                           =>

    google.com/anotherurl                            ==
    google.com/thisurl                               =>
    google.com/thisanotherurl                        =>
    google.com/anotherurl2                           =>

Where as i am more after

    InputObject                                 SideIndicator
    -----------                                 -------------
    google.com/thisurl                               ==
    google.com/thisanotherurl                        =>

    google.com/anotherurl                            ==
    google.com/anotherurl2                           =>

Hopefully that makes sense.

Is it possible to only do the compares only on the values where the ids match?

Please let me know if you have any further questions.

Thanks.

Answer 1

The problem was your iteration logic, which mistakenly processed multiple rules from the XML document in a single iteration:

• foreach ($xmlRule in $ruleSet) didn't enumerate anything - instead it processed the single <ruleset> element; to enumerate the child <rule> elements, you must use $ruleSet.rule.

• $xmlRule.rule.exceptions.exception.url then implicitly iterated over all <rule> children and therefore reported the URLs across all of them, which explains the extra lines in your Compare-Object output.

Here's a streamlined, annotated version of your code:

$existingWaf = Get-Content -LiteralPath stack.json | ConvertFrom-Json
$xmlFile = [xml] (Get-Content -raw -LiteralPath stack.xml )

# No need for a loop; $xmlFile is a single [System.Xml.XmlDocument] instance.
$ruleSet = $xmlFile.waf.ruleset

foreach ($siteRule in $ExistingWaf.security.waf.rules)
{
    # !! Note the addition of `.rule`, which ensures that the rules
    # !! are enumerated *one by one*. 
    foreach ($xmlRule in $ruleSet.rule)
    {
        if ($xmlRule.id -eq $siteRule.id)
        {
          # !! Note: `$xmlRule` is now a single, rule, therefore:
          # `$xmlRule.rule.[...]-> `$xmlRule.[...]`
          # Also note that neither @(...) nor Select-Object are needed, and
          # the `| where ...` (Where-Object) is not needed.
          Compare-Object -ReferenceObject $siteRule.exceptions.values.urls.value `
                         -DifferenceObject $xmlRule.exceptions.exception.url -IncludeEqual
        }
    }
}

---

Additional observations regarding your code:

• There is no need to ensure that operands passed to Compare-Object are arrays, so there's no need to wrap them in array sub-expression operator @(...). Compare-Object handles scalar operands fine.

• ... | Select-Object is a virtual no-op - the input object is passed through^[1]

• ... | Where-Object {$xmlRule.rule.id -eq $siteRule.id} is pointless, because it duplicates the enclosing foreach loop's condition.

  • Generally speaking, because you're not referencing the pipeline input object at hand via automatic variable $_, your Where-Object filter is static and will either match all input objects (as in your case) or none.

---

^{[1] There is a subtle, invisible side effect that typically won't make a difference: Select-Object adds an invisible [psobject] wrapper around the input object, which on rare occasions does cause different behavior later - see this GitHub issue.}