PowerShell Streaming Output

Question

I'd like to capture some streaming output in PowerShell. For example

cmd /c "echo hi && foo"

This command should print hi and then bomb. I know that I can use -ErrorVariable:

Invoke-Command { cmd /c "echo hi && foo" } -ErrorVariable ev

however there is an issue: in the case of long running commands, I want to stream the output, not capture it and only get the stderr/stdout output at the end of the command

Ideally, I'd like to be able to split stderr and stdout and pipe to two different streams - and pipe the stdout back to the caller, but be prepared to throw stderr in the event of an error. Something like

$stdErr
Invoke-Command "cmd" "/c `"echo hi && foo`"" `
  -OutStream (Get-Command Write-Output) `
  -ErrorAction {
    $stdErr += "`n$_"
    Write-Error $_
  }

if ($lastexitcode -ne 0) { throw $stdErr}

the closest I can get is using piping, but that doesn't let me discriminate between stdout and stderr so I end up throwing the entire output stream

function Invoke-Cmd {
<# 
.SYNOPSIS 
Executes a command using cmd /c, throws on errors.
#>
    param([string]$Cmd)
    )
    $out = New-Object System.Text.StringBuilder
    # I need the 2>&1 to capture stderr at all
    cmd /c $Cmd '2>&1' |% {
        $out.AppendLine($_) | Out-Null
        $_
    }

    if ($lastexitcode -ne 0) {
        # I really just want to include the error stream here
        throw "An error occurred running the command:`n$($out.ToString())" 
    }
}

Common usage:

Invoke-Cmd "GitVersion.exe" | ConvertFrom-Json

Note that an analogous version that just uses a ScriptBlock (and checking the output stream for [ErrorRecord]s isn't acceptable because there are many programs that "don't like" being executed directly from the PowerShell process

The .NET System.Diagnostics.Process API lets me do this...but I can't stream output from inside the stream handlers (because of the threading and blocking - though I guess I could use a while loop and stream/clear the collected output as it comes in)

Answer 1

<sup>- The behavior described applies to running PowerShell in regular console/terminal windows with no remoting involved. With remoting and in the ISE, the behavior is different as of PSv5.1 - see bottom.
- The 2>$null behavior that Burt's answer relies on - 2>$null secretly still writing to PowerShell's error stream and therefore, with $ErrorActionPreference Stop in effect, aborting the script as soon as an external utility writes anything to stderr - has been classified as a bug and is likely to go away.</sup>

• When PowerShell invokes an external utility such as cmd, its stderr output is passed straight through by default. That is, stderr output prints straight to the console, without being included in captured output (whether by assigning to a variable or redirecting to a file).

• While you can use 2>&1 as part of the cmd command line, you won't be able to distinguish between stdout and stderr output in PowerShell.

• By contrast, if you use 2>&1 as a PowerShell redirection, you can filter the success stream based on the input objects' type:

  • A [string] instance is a stdout line
  • A [System.Management.Automation.ErrorRecord] instance is a stderr line.

The following function, Invoke-CommandLine, takes advantage of this:

• Note that the cmd /c part isn't built in, so you would invoke it as follows, for instance:

  Invoke-CommandLine 'cmd /c "echo hi && foo"'
  

• There is no fundamental difference between passing invocation of a cmd command line and direct invocation of an external utility such as git.exe, but do note that only invocation via cmd allows use of multiple commands via operators &, &&, and ||, and that only cmd interprets %...%-style environment-variable references, unless you use --%, the stop-parsing symbol.

• Invoke-CommandLine outputs both stdout and stderr line as they're being received, so you can use the function in a pipeline.

  • As written, stderr lines are written to PowerShell's error stream using Write-Error as they're being received, with a single, generic exception being thrown after the external command terminates, should it report a nonzero $LASTEXITCODE.

  • It's easy to adapt the function:

    • to take action once the first stderr line is received.
    • to collect all stderr lines in a single variable
    • and/or, after termination, to take action if any stderr input was received, even with $LASTEXITCODE reporting 0.

• Invoke-CommandLine uses Invoke-Expression, so the usual caveat applies: be sure you know what command line you're passing, because it will be executed as-is, no matter what it contains.

---

function Invoke-CommandLine {
<# 
.SYNOPSIS 
Executes an external utility with stderr output sent to PowerShell's error           '
stream, and an exception thrown if the utility reports a nonzero exit code.   
#>

  param([parameter(Mandatory)][string] $CommandLine)

  # Note that using . { ... } is required around the Invoke-Expression
  # call to ensure that the 2>&1 redirection works as intended.
  . { Invoke-Expression $CommandLine } 2>&1 | ForEach-Object {
    if ($_ -is [System.Management.Automation.ErrorRecord]) { # stderr line
      Write-Error $_  # send stderr line to PowerShell's error stream
    } else { # stdout line
      $_              # pass stdout line through
    } 
  }

  # If the command line signaled failure, throw an exception.
  if ($LASTEXITCODE) {
    Throw "Command failed with exit code ${LASTEXITCODE}: $CommandLine"
  }

}

---

Optional reading: how calls to external utilities fit into PowerShell's error handling

^{Current as of: Windows PowerShell v5.1, PowerShell Core v6-beta.2}

• The value of preference variable $ErrorActionPreference only controls the reaction to errors and .NET exceptions that occur in PowerShell cmdlet/function calls or expressions.

• Try / Catch is for catching PowerShell's terminating errors and .NET exceptions.

• In a regular console window with no remoting involved, external utilities such as cmd currently never generate either error - all they do is report an exit code, which PowerShell reflects in automatic variable $LASTEXITCODE, and automatic variable $? reflects $False if the exit code is nonzero.

  • Note: The fact that the behavior differs fundamentally in hosts other than the console host - which includes the Windows ISE and when remoting is involved - is problematic: There, calls to external utilities result in stderr output treated as if non-terminating errors had been reported; specifically:

    • Every stderr output line is output as an error record and also recorded in the automatic $Error collection.
    • In addition to $? being set to $false with a nonzero exit code, the presence of any stderr output also sets it to $False.
    • This behavior is problematic, as stderr output by itself does not necessarily indicate an error - only a nonzero exit code does.
    • Burt has created an issue in the PowerShell GitHub repository to discuss this inconsistency.

  • By default, stderr output generated by an external utility is passed straight through to the console - they are not captured by PowerShell variable assignments or (success-stream) output redirections.

  • As discussed above, this can be changed:

    • 2>&1 as part of a command line passed to cmd sends stdout and stderr combined to PowerShell's success stream, as strings, with no way to distinguish between whether a given line was a stdout or stderr line.

    • 2>&1 as a PowerShell redirection sends stderr lines to PowerShell's success stream too, but you can distinguish between stdout- and stderr-originated lines by their DATA TYPE: a [string]-typed line is a stdout-originated line, whereas a [System.Management.Automation.ErrorRecord]-typed line is a stderr-originated one.

Answer 2

Note: updated sample below should now work across PowerShell hosts. GitHub issue Inconsistent handling of native command stderr has been opened to track the discrepancy in previous example. Note however that as it depends on undocumented behavior, the behavior may change in the future. Take this into consideration before using it in a solution that must be durable.

You are on the right track with using pipes, you probably don't need Invoke-Command, almost ever. Powershell DOES distinguish between stdout and stderr. Try this for example:

cmd /c "echo hi && foo" | set-variable output

The stdout is piped on to set-variable, while std error still appears on your screen. If you want to hide and capture the stderr output, try this:

cmd /c "echo hi && foo" 2>$null | set-variable output

The 2>$null part is an undocumented trick that results in the error output getting appended to the PowerShell $Error variable as an ErrorRecord.

Here's an example that displays stdout, while trapping stderr with an catch block:

function test-cmd {
    [CmdletBinding()]
    param()

    $ErrorActionPreference = "stop"
    try {
        cmd /c foo 2>$null
    } catch {
        $errorMessage = $_.TargetObject
        Write-warning "`"cmd /c foo`" stderr: $errorMessage"
        Format-list -InputObject $_ -Force | Out-String | Write-Debug
    }
}

test-cmd

Generates the message:

WARNING: "cmd /c foo" stderr: 'foo' is not recognized as an internal or external command

If you invoke with debug output enabled, you'll alsow see the details of the thrown ErrorRecord:

DEBUG: 

Exception             : System.Management.Automation.RemoteException: 'foo' is not recognized as an internal or external command,
TargetObject          : 'foo' is not recognized as an internal or external command,
CategoryInfo          : NotSpecified: ('foo' is not re...ternal command,:String) [], RemoteException
FullyQualifiedErrorId : NativeCommandError
ErrorDetails          : 
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at test-cmd, <No file>: line 7
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {}
PSMessageDetails      : 

Setting $ErrorActionPreference="stop" causes PowerShell to throw an exception when the child process writes to stderr, which sounds like it's the core of what you want. This 2>$null trick makes the cmdlet and external command behavior very similar.