Synapse/ADF "errorCode": "2200", "message": "Failure happened on 'Source' side. ErrorCode=JsonInvalidDataFormat

Question

I am making a rest call using a copy activity to write data to a datalake storage. Source is a rest dataset and sink is a json. If I use a http binary dataset it works fine, but then I can not perform pagination in an easy way. I am getting the following error:

{
    "errorCode": "2200",
    "message": "Failure happened on 'Source' side. ErrorCode=JsonInvalidDataFormat,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error occurred when deserializing source JSON file ''. Check if the data is in valid JSON object format.,Source=Microsoft.DataTransfer.ClientLibrary,'",
    "failureType": "UserError",
    "target": "ingest json to landing",
    "details": []
}

If I only preview the data in adf I am getting the following error:

Error code 21155 Details Error occurred when deserializing source JSON file ''. Check if the data is in valid JSON object format.

If I make the same call with postman, no issues, but the body comes back as text and looks as follows: enter image description here

[
   {
      "Data":1561,
      "Number":156,
      "ID":1565,
      "TypeID":15,
      "Type":"sdfg",
      "Comments":"jbgijdresbgiesugbiiergburesgbiergb breijfberiugbi iuergiuer erguiergeriu erreuguierh guierhger eu u geir er erug iug    eruhiuguiergiuguer  er ug e  eruhgeuirge erug  erugeurhgeueruig .\nTips\n1.\trjgnrig reigeirg  eirghi : \nall erjgeriugher ergierig I er gheh erh eirghherigerg ger er ghiergier  eh egrigerig  eg . \n2.   Ewgheri ughergh erh  r   erhgerh:\ergjoi ihg iergierg iererigiergi er gieriger er gier iergpai g aei[g rhe agieg i[e I erg hhg rggergieerig ei gerh ergio ehaigneigrdrg dosg .\n3.  Jsbdiujasbfib:\npay erkgierogh erigerho ig er  erh oegiuerghe g e ir oego gerghierghe  ge rgerihgeri gegh eregh.\n"
   },
{
      "Data":1561,
      "Number":156,
      "ID":1565,
      "TypeID":15,
      "Type":"sdfg",
      "Comments":"jbgijdresbgiesugbiiergburesgbiergb breijfberiugbi iuergiuer erguiergeriu erreuguierh guierhger eu u geir er erug iug    eruhiuguiergiuguer  er ug e  eruhgeuirge erug  erugeurhgeueruig .\nTips\n1.\trjgnrig reigeirg  eirghi : \nall erjgeriugher ergierig I er gheh erh eirghherigerg ger er ghiergier  eh egrigerig  eg . \n2.   Ewgheri ughergh erh  r   erhgerh:\ergjoi ihg iergierg iererigiergi er gieriger er gier iergpai g aei[g rhe agieg i[e I erg hhg rggergieerig ei gerh ergio ehaigneigrdrg dosg .\n3.  Jsbdiujasbfib:\npay erkgierogh erigerho ig er  erh oegiuerghe g e ir oego gerghierghe  ge rgerihgeri gegh eregh.\n"
   }
]

This is how the copy activity is configured:

Does anybody know if a rest dataset supports array of jsonlines?

If you put the JSON into a validator, it shows an error: https://www.online-json.com/json-validator It appears to return "undefined" for the Comments property. If you remove the Comments content, the JSON validates. — Joel Cochran, Oct 31 '22 at 13:14
The json example I made a manually mistake. I fixed the example, any ideas why the pipeline is failing? — Abashirsoldom, Oct 31 '22 at 18:44
Anybody knows why with http it works, but with a rest source it is giving the deserializing error? — Abashirsoldom, Nov 02 '22 at 08:04
Did you update the sample code in the question? Because I'm still getting a parse error with a "Bad String" message in the validator. — Joel Cochran, Nov 02 '22 at 13:18
Hi @Abashirsoldom, can you please edit and provide snippet what you have done in postman . — B. B. Naga Sai Vamsi, Nov 04 '22 at 06:39

score 0 · Answer 1 · answered Nov 02 '22 at 20:24

0

This is what i did , I took the JSON which you had pasted above . I used that JSON as source and check how ADF handled that .When i tried to preview that data in ADF ( configured the JSON as source) , it failed . I think that the character "\e" is for some reason breaking the JSON . I have no idea at this time , why thats happening

answered Nov 02 '22 at 20:24

HimanshuSinha

1,650
2
6
10

I will fix the **example**. But here is the thing, with postman no issues and the other thing os that adf is giving the exact error even if the body is **empty**. – Abashirsoldom Nov 03 '22 at 07:23

score 0 · Answer 2 · answered Nov 04 '22 at 09:44

Note: When you are using a binary dataset in copy activity. you only copy from binary dataset to binary dataset otherwise it will throw an error.

Source:

Request method : GET

Add additional header : Authorization : Bearer < Token >

enter image description here

Sink:

If you want multiple responses into a single file it will break the JSON format and lead to invalid JSON. Please change the file pattern to an Array of object or Set of object.

enter image description here

Output:

enter image description here

For more details refer this SO thread and MS Document

Thx for the explanation. I tried that, but still got the same error. The error states that the problem is in the source. — Abashirsoldom, Nov 05 '22 at 07:22

Synapse/ADF "errorCode": "2200", "message": "Failure happened on 'Source' side. ErrorCode=JsonInvalidDataFormat

2 Answers2