0

I read a big Json into powershell and afterwards reduce the fields via

select-object -Property $myProperties

Its running fine, but its slow.

Are there more performant alterantives out there?

I read a little bit about linq, but I am not sure if this will resolve my issues. I have also thought about reading the com ntent as hashtable via the -ashashtable operator from convertfrom-json and then removing the unnessecary properties via the .remove operator but it needs additional processing for arrays.

Does anyone have a general advice, what's the preferred solution?

Full Script

function Start-TestDatabaseFile {
    param (
        $folderSoll,
        $folderIst
    )

    $workfolderPath = $workfolder.path

    $errors = 0

    $dbInfo = Get-dbInformation

    $localFolder = "$workfolderPath\TestResults\DatabaseFileDifference"
    New-Item $localFolder -ItemType Directory | Out-Null

    $differencePerTable = "$workfolderPath\TestResults\DatabaseFilePerTable"
    New-Item $differencePerTable -ItemType Directory | Out-Null

    $keys = $result.Keys | Sort-Object

    foreach ($key in $keys) {
        #region File Datenmodell
        $firstItemFile = $result[$key].fileRaw.Replace('_response.json', '_Datenbank.json')

        $tf = $result[$key].tf
        $myTestfall = $myTest[$tf].root

        if ($null -ne $myTestfall) {
            $responseFile = Split-Path $firstItemFile -Leaf
            $myPath = $result[$key].myPath
            $file = Join-Path $myTest[$myTestfall].folder -ChildPath ("$myPath\$responseFile")
            $secondItemFile = $file.Replace('\Requests\', '\Responses_SOLL\')
        }
        else {
            $secondItemFile = $firstItemFile.replace($folderIst, $folderSoll)
        }

        try {
            $firstItemAll = Get-Content $firstItemFile -ErrorAction Stop | ConvertFrom-Json -AsHashtable
            $firstItem = $firstItemAll.post
            $firstItemExists = $true
        }
        catch {
            $firstItem = '{}' | ConvertFrom-Json -AsHashtable
            $firstItemExists = $false
        }

        try {
            $secondItemAll = Get-content $secondItemFile -ErrorAction Stop | ConvertFrom-Json -AsHashtable
            $secondItem = $secondItemAll.post
            $secondItemExists = $true
        }
        catch {
            $secondItem = '{}' | ConvertFrom-Json -AsHashtable
            $secondItemExists = $false
        }

        $views = $dbInfo.keys | Where-Object { $_ -like 'V*' }

        foreach ($view in $views) {
            $schalter = $dbInfo[$view].myItem
            if ($firstItemExists) {
                try {
                    $firstItem.$view = [array]($firstItem.$view | Select-Object -Property $dbInfo[$view].fieldsCompare)
                }
                catch {
                    [console]::WriteLine('Issue in Start-TestDatabaseFile ExceptionPoint1')
                }
            }

            if ($secondItemExists) {
                if ($null -ne $myTestfall) {
                    $secondItem.$view = $secondItemAll.pre.$view
                }

                if ($useNewDm.$schalter) {
                    $secondItem.$view = [array]($secondItem.$view | Select-Object -Property $dbInfo[$view].fieldsCompare)
                }
                else {
                    $secondItem.$view = [array]($firstItemAll.pre.$view | Select-Object -Property $dbInfo[$view].fieldsCompare)
                }
            }
        }

        $firstItemJson = $firstItem | ConvertTo-Json -Depth 100
        $secondItemJson = $secondItem | ConvertTo-Json -Depth 100

        $result[$key].resultDatabaseFile = Compare-Json -ist $firstItemJson -soll $secondItemJson
        $result[$key].passedDatabaseFile = 'passed'

        if ($null -ne $result[$key].resultDatabaseFile.difference) {
            $filename = $key.replace('\', '_')

            $result[$key].resultDatabaseFile.difference | Out-File -FilePath ("$localFolder\$filename.json")

            #region tmp out
            $difference = $result[$key].resultDatabaseFile.difference | ConvertFrom-Json
            $diffKeys = (Get-Member -InputObject $difference -MemberType NoteProperty).Name

            foreach ($diffKey in $diffKeys) {

                $file = "$differencePerTable\$diffKey.json"

                Add-Content -Path $file -Value $key
                Add-Content -Path $file -Value ($difference.$diffkey | ConvertTo-Json)
            }
            #endregion

            $result[$key].passedDatabaseFile = 'failed'

            $errors += 1
        }
        #endregion
    }

    return $errors
}
Mathias R. Jessen
  • 157,619
  • 12
  • 148
  • 206
Farbkreis
  • 604
  • 3
  • 12
  • 2
    What makes you believe that `Select-Object` is the bottleneck? Have you measured how fast it runs with vs. without `Select-Object`? Where do the property names in `$myProperties` coming from? Are you attempting to discover them dynamically? This will probably be a lot easier if you post your whole script :-) – Mathias R. Jessen Oct 04 '21 at 12:43
  • Yeah I have also checked it with a profiler, but anyway will post the script. – Farbkreis Oct 04 '21 at 12:44
  • I'm not sure what `Get-DbInformation` returns, but could it be that powershell is querying your database to enumerate `$dbInfo[$view].fieldsCompare` ? Try setting the fields text to a variable once, then use `Select $MyFields` instead for your `Select-Object` statements. – Cpt.Whale Oct 04 '21 at 21:13

0 Answers0