0

I am running a query in Stream analytics with a UNION function. I would like to get the distinct values for the query results. Since the UNION is allowing duplicates in Azure Stream Analytics I am getting result with duplicate values. I have tried using DISTINCT keyword also but even it is not working.

Below is the Query I tried.

    WITH 
     ABCINNERQUERY AS (
     SELECT
        event.ID as ID,
        ABCArrayElement.ArrayValue.E as TIME,
        ABCArrayElement.ArrayValue.V as ABC
    FROM
        [YourInputAlias] as event
    CROSS APPLY GetArrayElements(event.ABC) AS ABCArrayElement
    ),
     XYZINNERQUERY AS (
      SELECT
        event.ID as ID,
        XYZArrayElement.ArrayValue.E as TIME,
        XYZArrayElement.ArrayValue.V as XYZ
    FROM
        [YourInputAlias] as event
    CROSS APPLY GetArrayElements(event.XYZ) AS XYZArrayElement
),
KEYS AS
(
    SELECT DISTINCT
        ABCINNERQUERY.ID AS ID, 
        ABCINNERQUERY.TIME as TIME
    FROM ABCINNERQUERY
        UNION 
    SELECT DISTINCT
        XYZINNERQUERY.ID AS ID, 
        XYZINNERQUERY.TIME as TIME
    FROM XYZINNERQUERY
)

SELECT
    KEYS.ID as ID,
    KEYS.TIME as TIME

INTO [YourOutputAlias] 
FROM KEYS

In the above query ID is unique and Time will be An array of values with Time and value of ABC/XYZ.

input json file is as below.

[
{"ID":"006XXXXX",
"ABC":
[{"E":1557302231320,"V":54.799999237060547}],
"XYZ":
[{"E":1557302191899,"V":31.0},{"E":1557302231320,"V":55}],

{"ID":"007XXXXX",
"ABC":
[{"E":1557302195483,"V":805.375},{"E":1557302219803,"V":0}],
"XYZ":
[{"E":1557302219803,"V":-179.0},{"E":1557302195483,"V":88}]
]

Expected result without duplicates.

Antony
  • 970
  • 3
  • 20
  • 46

0 Answers0