PowerShell Write-Output vs variable dumping in a piping operation

Question

I have a function that converts a PSObject into a hashtable. Function works well, but there's a little subtlety that I am trying to understand and can't really grasp my head around.

I'm using PowerShell Core 7.0.3

The function:

function Convert-PSObjectToHashtable
{
    param (
        [Parameter(ValueFromPipeline)]
        $InputObject
    )

    process
    {
        if ($null -eq $InputObject) { return $null }

        if ($InputObject -is [System.Collections.IEnumerable] -and $InputObject -isnot [string])
        {
            $collection = @(
                foreach ($object in $InputObject) { Convert-PSObjectToHashtable $object }
            )

            # buggy
            #Write-Output -NoEnumerate $collection
            
            # correct
            $collection
        }
        elseif ($InputObject -is [psobject])
        {
            $hash = @{}

            foreach ($property in $InputObject.PSObject.Properties)
            {
                $hash[$property.Name] = Convert-PSObjectToHashtable $property.Value
            }

            $hash
        }
        else
        {
            $InputObject
        }
    }
}

I execute the following code:

$obj = "{level1: ['e','f']}"
$x = $obj | ConvertFrom-Json | Convert-PSObjectToHashtable
[Newtonsoft.Json.JsonConvert]::SerializeObject($x)

The "buggy" code returns me:

{"level1":{"CliXml":"<Objs Version=\"1.1.0.1\" xmlns=\"http://schemas.microsoft.com/powershell/2004/04\">\r\n  <Obj RefId=\"0\">\r\n    <TN RefId=\"0\">\r\n      <T>System.Object[]</T>\r\n      <T>System.Array</T>\r\n      <T>System.Object</T>\r\n    </TN>\r\n    <LST>\r\n      <S>e</S>\r\n      <S>f</S>\r\n    </LST>\r\n  </Obj>\r\n</Objs>"}}

The correct code returns me:

{"level1":["e","f"]}

Why wouldn't the buggy code work, if technically, in PowerShell when working with the object result, they look equivalent?

Thank you!

Answer 1

This occurs because PowerShell loves to wrap things in PSObject's.

The "plumbing" through which Write-Output (and all other binary cmdlets) emit standard output is implemented in a way that forces this explicit wrapping of the input objects in a PSObject.

So from a PowerShell user's perspective, these two variables have identical values:

$a = 1..3 |Write-Output
$b = 1..3

By any reasonable indication, both variables hold an array containing the integers 1,2,3:

PS ~> $a.GetType().Name
Object[]
PS ~> $b.GetType().Name
Object[]
PS ~> $a[0] -is [int]
True
PS ~> $a[0] -eq $b[0]
True

Behind the scenes though, the object hierarchy actually looks like this:

$a = 1..3 |Write-Output
# Behaves like: @(1,2,3)
# Is actually:  @([psobject]::new(1),[psobject]::new(2),[psobject]::(3))

$b = 1..3
# Behaves like: @(1,2,3)
# Is actually : @(1,2,3)

You might think this would pose a problem, but PowerShell goes through great length to keep this wrapper layer completely hidden from the user. When the runtime subsequently evaluates a statement like $a[1] and finds a PSObject wrapper, it transparently returns the base value (e.g. 2) as if it was the actual value referenced by the underlying array.

But [JsonConvert]::SerializeObject() isn't written in PowerShell, and when it starts traversing the object hierarchy outside the confines of the PowerShell language engine, it encounters the wrapping PSObject instances and picks its default serialization format (CliXml) instead of what should have otherwise been treated as native JSON types.

The expression $collection on the other hand is not a binary cmdlet and there are no downstream pipeline consumers, so its value is enumerated and written directly to the output stream, bypassing the PSObject wrapping/boxing step. The resulting array therefore references the output values directly instead of their respective PSObject wrappers, and the serialization works as expected again.

---

You can unwrap objects by referencing the ImmediateBaseObject property on the hidden psobject memberset:

$a = 1,2 |Write-Output
# Actual: @([psobject]::new(1),[psobject]::new(2))

$a = $a |ForEach-Object { $_.psobject.ImmediateBaseObject }
# Actual: @(1,2)

Beware that wrapping re-occurs every time an object goes through |:

$a = 1,2
# Actual: @(1,2)

$a = $a |ForEach-Object { $_ }
# Actual: @([psobject]::new(1),[psobject]::new(2))

If you wonder whether an expression returns PSObject-wrapped objects from within PowerShell, pass the output to Type.GetTypeArray():

PS ~> [type]::GetTypeArray(@(1..3|Write-Output)).Name
PSObject
PSObject
PSObject
PS ~> [type]::GetTypeArray(@(1..3)).Name
Int32
Int32
Int32