0

I successfully sent an webrequest to a Website and got a responde:

$Uri = "https://URLXYZ"

$Method = "POST"

$Header = @{
    "Accept" = "*/*";
    "Connection" = "keep-alive";
    "Accept-Encoding" = "gzip, deflate, br";
    "Content-Type" = "text/json; charset=""UTF-8"""
}

$Body = @"
{
    "Items": [
        {
            "Type": "XX",
            "Value": "YY",
        }
    ],
    "TypeId": XY,
    "LiveConditions": []
}
"@

$webrequest = Invoke-WebRequest -Uri $Uri -Body $Body -Headers $Header -Method $Method

Now, I am trying to convert it from JSON without success:

$webrequest.Content | ConvertFrom-Json
ConvertFrom-Json: Conversion from JSON failed with error: Unexpected character encountered while parsing value: . Path '', line 0, position 0.

When I copy the output ($webrequest.Content) to Notepad++ I can see a carriage return (CR) and line feed (LF):

$webrequest.Content

[{"MailG":[{"DisplayName":{"7":"Worker","9":"Person"},"Mails":"max.mustermann@gmai.com;"}],"ResultCount":1,"Rows":[{"ElementGroups":[{"ObjectContainer":{"Id":55
6677889900,"UID":"1122334455","Info":[],"PreCalculatedSelectedProperties":{"11":"Mustermann","22":"","33":"StreetName","44":"","55":"max.mustermann@gmai.com","66":"","77":"Berlin","88":"","99":"Max Mustermann","00":"+49 00 000 000","111":"userid","222":"xyz","333":"company","444":"1122334455","555":"roomnumber","666":"Germany","777":"Team","888":"+49 000 0 00 0","999":"Max","000":""},"TID":5,"RuleConditionPartResults":{"1448925":false,"1448926":false,"1448927":false,"1448928":false,"1448929":false,"1448930":false,"1448931":false,"1448932":false,"1448933":false,"1448934":false,"1448935":false,"1448936":false,"1448937":false,"1448938":false,"1448939":false,"1448940":false,"1448941":false,"1448942":false},"Img":{"3714":["picture"]},"Parents":[],"Childs":[],"UpObjects":0,"Down":0,"LinkCount":0,"FootObject":{},"BoxIds":[],"DisplayValue":"Max Mustermann","Key":"1122334455"},"Columns":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]}],"Blank":{},"Score":0.0,"SInd":0}],"Page":0,"TID":5}]

enter image description here

This is unexpected cause the CR;LF is e.g. in the middle of an email address. If the response is longer, there are multiple CR;LF within the content of the webrequest. The $webrequest.content is TypeName: System.String

If I manually remove the CR;CF in Notepad++ and PrettyPrint it is working:

[
    {
        "MailG": [
            {
                "DisplayName": {
                    "7": "Worker",
                    "9": "Person"
                },
                "Mails": "max.mustermann@gmai.com;"
            }
        ],
        "ResultCount": 1,
        "Rows": [
            {
                "ElementGroups": [
                    {
                        "ObjectContainer": {
                            "Id": 556677889900,
                            "UID": "1122334455",
                            "Info": [],
                            "PreCalculatedSelectedProperties": {
                                "11": "Mustermann",
                                "22": "",
                                "33": "StreetName",
                                "44": "",
                                "55": "max.mustermann@gmai.com",
                                "66": "",
                                "77": "Berlin",
                                "88": "",
                                "99": "Max Mustermann",
                                "00": "+49 00 000 000",
                                "111": "userid",
                                "222": "xyz",
                                "333": "company",
                                "444": "1122334455",
                                "555": "roomnumber",
                                "666": "Germany",
                                "777": "Team",
                                "888": "+49 000 0 00 0",
                                "999": "Max",
                                "000": ""
                            },
                            "TID": 5,
                            "RuleConditionPartResults": {
                                "1448925": false,
                                "1448926": false,
                                "1448927": false,
                                "1448928": false,
                                "1448929": false,
                                "1448930": false,
                                "1448931": false,
                                "1448932": false,
                                "1448933": false,
                                "1448934": false,
                                "1448935": false,
                                "1448936": false,
                                "1448937": false,
                                "1448938": false,
                                "1448939": false,
                                "1448940": false,
                                "1448941": false,
                                "1448942": false
                            },
                            "Img": {
                                "3714": [
                                    "picture"
                                ]
                            },
                            "Parents": [],
                            "Childs": [],
                            "UpObjects": 0,
                            "Down": 0,
                            "LinkCount": 0,
                            "FootObject": {},
                            "BoxIds": [],
                            "DisplayValue": "Max Mustermann",
                            "Key": "1122334455"
                        },
                        "Columns": [
                            0,
                            1,
                            2,
                            3,
                            4,
                            5,
                            6,
                            7,
                            8,
                            9,
                            10,
                            11,
                            12,
                            13,
                            14,
                            15,
                            16,
                            17
                        ]
                    }
                ],
                "Blank": {},
                "Score": 0.0,
                "SInd": 0
            }
        ],
        "Page": 0,
        "TID": 5
    }
]

I already tried to convert it or even replace the CR;LF it in PowerShell without success.

The only workaround I found is to write the content to a .txt-file and read it again.

$webrequest.Content | Out-File "C:\Temp\WebRequestTemp.txt"
$json = Get-Content "C:\Temp\WebRequestTemp.txt" | ConvertFrom-Json

Afterwards I was able to convert it from JSON and work with the data.

Why can I not directly convert it from JSON as usual?

kaiaschulz
  • 35
  • 1
  • 6
  • We cannot see the (sanitized) contents, but if as you say there are newline characters in there you could try to remove them all. `$webrequest.Content -replace '\r?\n' | ConvertFrom-Json`. If this breaks the content even further or not, we have no idea without you showing us that content.. – Theo Jan 26 '22 at 12:07
  • Unfortunately, I already tried this as well without success. Nevertheless, I impersonalized the **$webrequest.content** and added it to the main post – kaiaschulz Jan 26 '22 at 13:21
  • Just curious as it is giving a JSON response back it may be worth using `Invoke-RestMethod` as opposed to `Invoke-WebRequest`. If this is successful you wont need to `ConvertFrom-JSON` as this is done by the restmethod cmdlet. (Providing it does not fail with the same issue.) – CraftyB Jan 26 '22 at 14:06
  • That's weird, because if I try your example with `$webrequest.content -replace '\r?\n' | ConvertFrom-Json` it works. Even after slamming in more newlines (`CRLF`) at random places, the code works. If I convert that back to JSON with `ConvertTo-Json -Depth 7` I get valid JSON... – Theo Jan 26 '22 at 14:07
  • `Invoke-RestMethod` could be used as well and I already tried it with the same result. ```$webrequest = Invoke-RestMethod -Uri $Uri -Body $Body -Headers $Header -Method $webrequest | ConvertFrom-Json ConvertFrom-Json: Conversion from JSON failed with error: Unexpected character encountered while parsing value: . Path '', line 0, position 0```. With other request I never had problems with the converting and still can do it without any problems :( – kaiaschulz Jan 26 '22 at 14:17
  • Can you please confirm the type of `$webrequest.Content` by using `$webrequest.Content.GetType().Name`, In theory this should output `String`, if not then this is where your problem will be. – CraftyB Jan 27 '22 at 10:39
  • Just checked it. The output of `$webrequest.Content.GetType().Name` is `String`. – kaiaschulz Jan 27 '22 at 10:46

2 Answers2

1

I had a similar issue, and the issue was there was an unknown invisible character at the beginning of the content block. This character was "U+FEFF".

I managed to get it to be replaced by doing $webrequest.Content -replace '\uFEFF' | ConvertFrom-Json

I hope this works for anyone else too.

thakyZ
  • 113
  • 1
  • 2
  • 12
0

To help you with a more detailed answer. It is useful to share the contents of the web response. But maybe the line below will solve your problem.

$jsonCorrected = [Text.Encoding]::UTF8.GetString([Text.Encoding]::GetEncoding(28591).GetBytes(($webrequest.Content)))
R.Amersfoort
  • 85
  • 3
  • Unfortunately, the full web response is confidential. Nevertheless, the CR;LF is here: "Un _(CR;LF)_ iqueDataId": The problem with your suggestion is still the same: ```$jsonCorrected = [Text.Encoding]::UTF8.GetString([Text.Encoding]::GetEncoding(28591).GetBytes(($webrequest.Content)))``` ```$jsonCorrected | ConvertFrom-Json``` ```ConvertFrom-Json: Conversion from JSON failed with error: Unexpected character encountered while parsing value: ?. Path '', line 0, position 0.``` – kaiaschulz Jan 26 '22 at 09:31