-1

I've been looking around the web for the past week to solve this issue, but I'm almost sure now that there is no solution, so this is a good challenge for many of you I guess :)

In a prod environment we are using powershell scripts to automate some processes of configuration and maintenance. These scripts always worked, until the new versions of the command line application (the scripts work with a CLI version of a bigger application), started outputting the XML results as UTF8.

This broke every script that was using this functionality.

The problem is that now there is a space, or null byte (sorry but I'm not very confident on this topic) between every single character, when redirecting output of this application to a text/log file.

I've already tried all the solutions already provided to many similar questions internet-wide, setting encoding to ascii, unicode, utfXX, OEM with the use of Set-Content, Out-File and all this sort of workarounds (even Out-String): the file is always containing these weird spaces, even two spaces when using some specific encodings.

Environment details:

> chcp
Active code page: 437

> $outputencoding
IsSingleByte      : True
BodyName          : us-ascii
EncodingName      : US-ASCII
HeaderName        : us-ascii
WebName           : us-ascii
WindowsCodePage   : 1252
IsBrowserDisplay  : False
IsBrowserSave     : False
IsMailNewsDisplay : True
IsMailNewsSave    : True
EncoderFallback   : System.Text.EncoderReplacementFallback
DecoderFallback   : System.Text.DecoderReplacementFallback
IsReadOnly        : True
CodePage          : 20127

I'd really appreciate any help, suggestion, or even just some moral support

General Grievance
  • 4,555
  • 31
  • 31
  • 45
REMOVED
  • 1
  • 2
  • `[Console]::OutputEncoding=[Text.Encoding]::UTF8` or `Start-Process -RedirectStandardOutput` – user4003407 Nov 23 '16 at 11:39
  • 2
    Then you should post some code, or better [mcve]. There are many places where it can go wrong, but without code anyone can only guess. – user4003407 Nov 23 '16 at 17:36
  • Please [edit] your question rather than posting code snippets in comments. Follow @PetSerAl's comment, post a [mcve]. – JosefZ Nov 24 '16 at 09:30

1 Answers1

1

Default encoding for > Redirection as well as for Out-File is Unicode:

The Out-File cmdlet sends output to a file. You can use this cmdlet instead of the redirection operator (>) when you need to use its parameters.

-Encoding

Specifies the type of character encoding used in the file. Valid values are "Unicode", "UTF7", "UTF8", "UTF32", "ASCII", "BigEndianUnicode", "Default", and "OEM".
"Unicode" is the default.

"Default" uses the encoding of the system's current ANSI code page.

"OEM" uses the current original equipment manufacturer code page identifier for the operating system.

Sample script:

''
'$outputencoding>$env:TEMP\40763209.txt'
$outputencoding>$env:TEMP\40763209.txt
(Get-Content -Path $env:TEMP\40763209.txt -Encoding Byte -TotalCount 16 |
     ForEach-Object {"{0:x2}" -f $_}) -join ' '
''
'$outputencoding|out-file $env:TEMP\40763209a.txt -Encoding utf8'
$outputencoding|out-file $env:TEMP\40763209a.txt -Encoding utf8
(Get-Content -Path $env:TEMP\40763209a.txt -Encoding Byte -TotalCount 16 |
     ForEach-Object {"{0:x2}" -f $_}) -join ' '

Output: note Byte Order Mark signatures in both files:

PS D:\PShell> D:\PShell\SO\40763209.ps1

$outputencoding>$env:TEMP\40763209.txt
ff fe 0d 00 0a 00 0d 00 0a 00 49 00 73 00 53 00

$outputencoding|out-file $env:TEMP\40763209a.txt -Encoding utf8
ef bb bf 0d 0a 0d 0a 49 73 53 69 6e 67 6c 65 42
JosefZ
  • 28,460
  • 5
  • 44
  • 83
  • My issue is exactly after this step: I was able to identify the encoding by simply opening a text file with the redirected output and clicking on "Save As.." in Notepad. The problem is that converting the content of the file to **any** other encoding results in the same situation: spaces between each character, or even two spaces in some cases. Avoiding the BOM is something that I already tried by reading many other similar questions similar to mine on the internet. – REMOVED Nov 24 '16 at 08:08