How to render JSON using Stream Analytics Query

Question

I have Inputs in the form of JSON stored in Blob Storage I have Output in the form of SQL Azure table.

My wrote query and successfully moving value of specific property in JSON to corresponding Column of SQL Azure table.

Now for one column I want to copy entire JSON payload as Serialized string in one sql column , I am not getting proper library function to do that.

SELECT
     CASE 
        WHEN GetArrayLength(E.event) > 0
            THEN GetRecordPropertyValue(GetArrayElement(E.event, 0), 'name')
        ELSE ''
    END AS EventName 
    ,E.internal.data.id as DataId
    ,E.internal.data.documentVersion as DocVersion

    ,E.context.custom As CustomDimensionsPayload

Into OutputTblEvents
FROM InputBlobEvents E

This CustomDimensionsPayload should be a JSON actually

If your input json column "CustomDimensionsPayload" is an object (as opposed to a string), this is currently not possible in azure stream analytics. — Vignesh Chandramohan, Jan 09 '17 at 02:44

score 2 · Accepted Answer · edited Mar 07 '17 at 13:58

2

I made a user defined function which did the job for me:

function main(InputJSON) {
    var InputJSONString = JSON.stringify(InputJSON);
    return InputJSONString;
}

Then, inside the Query, I used the function like this:

SELECT udf.ConvertToJSONString(COLLECT()) AS InputJSON
INTO outputX
FROM inputY

edited Mar 07 '17 at 13:58

davenpcj

12,508
5
40
37

answered Mar 07 '17 at 13:22

Per Schjetne

46
4

Did you add a GROUP BY? It fails for me with the following error: Stream Analytics job has validation errors: Query compilation error: Function 'collect' is either not supported or not usable in this context. User defined function calls must start with "udf." prefix. Aggregate functions must be used together with a GROUP BY clause.. – Christian Weiss Apr 23 '18 at 06:45

score 0 · Answer 2 · answered Apr 09 '20 at 07:40

You need to just reference the input object itself instead of COLLECT() if you want the entire payload to be converted. I was trying to do this also so figured I'd add what i did.

I used the same function suggested by PerSchjetne, query then becomes

SELECT udf.JSONToString(IoTInputStream)
INTO [SQLTelemetry]
FROM [IoTInputStream]

Your output will now be the full JSON string, including all the metadata extras that IOT hub adds on.

How to render JSON using Stream Analytics Query

2 Answers2