0

We use Time Series Insights Gen 2 that reads data from the IoT Hub. The data has a very similar form as Example C in this documentation: https://learn.microsoft.com/en-us/azure/time-series-insights/concepts-json-flattening-escaping-rules

Our data has the following structure:

{
  "timestamp": "2020-11-01T10:00:00.000Z",
  "sensors": [{
        "name" : "temperature",
        "unit" : "celsius"
        "value": 25.39
    },
    {
        "name" : "humidity",
        "unit" : "percentage" 
        "value": 97.85
    }
  ]
}

And usuall with many more 'sensor' items in the array. We always used the Time Series Insights PAYG Preview, and it worked fine. We could query the temperature with the following JSON payload:

{
  "getEvents": {
    "timeSeriesId": ["some Id"],
    "searchSpan": {
        "from": "2020-08-27T07:34:00.000Z",
        "to": "2020-08-27T07:34:10.000Z"
    },
    "filter": {
        "tsx": "$event.sensors_name.String = 'temperature'"
    },
    "projectedProperties": [{
            "name": "sensors_value",
            "type": "Double"
        }]
    }
}

This worked perfectly, until Microsoft bumped the PREVIEW PAYG version towards an official release. It doesn't work anymore, and we've found this in the documentation:

enter image description here

source: https://learn.microsoft.com/en-us/azure/time-series-insights/concepts-supported-data-types

The following documentation even clearly states that the flattening of arrays are different now: enter image description here

source: https://learn.microsoft.com/en-us/azure/time-series-insights/ingestion-rules-update

Is there a possibility to still query this dynamic type of array, without having to put the timestamp or deviceId in each array item?

Rens Groenveld
  • 960
  • 11
  • 28

1 Answers1

1

That functionality is not yet there, dynamic types can't be referenced in TSX expressions and the values within the array cannot be retrieved as projectedProperties. The entire array would have to be retrieved and then parsed client-side. To trigger flattening you'd have to add an ID or timestamp within the objects as you mention above.

One idea--create a new TSI environment and configure a composite TS ID--whatever your initial ID is plus sensor.name, (assuming that name came be a unique identifier), then you'd also take away the need for the filter for temp.

ranah
  • 707
  • 1
  • 6
  • 11
  • "dynamic types are only exposed via the GetEvents API" What does this mean? How can they be exposed through the GetEventsAPI? The payload that I mention above is also a GetEvents payload, right? – Rens Groenveld Aug 27 '20 at 20:11
  • yes, you're right, I responded too quickly w/o really looking at your request body. Edited above. GetEvents--just getting it "raw" – ranah Aug 27 '20 at 21:27
  • https://learn.microsoft.com/en-us/azure/time-series-insights/how-to-select-tsid#example-2-time-series-id-with-a-composite-key – ranah Aug 27 '20 at 21:42
  • Thanks for the reply. Although that works, it would require to reinstall TSI for existing production environments. We might have to rebuild the structure of the incoming messages so a timestamp is included in each object... Or... can we keep current structure for existing environments? Do you know when they will be updated so that we wouldn't be able to use our current getEvents query anymore? I read that the API will be updated end of october, but does that mean the version of TSI will also be upgraded to official release? – Rens Groenveld Aug 28 '20 at 07:40
  • Hi, my apologies I had notifications turned off. The previous API version will be retired at the end of Oct, ($event['sensor_name'].String or $event.sensor_name.String but NOT $event.[sensor_name].String), but that is orthogonal to the ingestion rules--two different trains. Any environment made during Preview can (must) use the new API syntax. At some point, the Preview flattening rules will be retired, but there is no timeline currently. It falls under the scope of an Azure breaking change and as such you'll have ample time to prepare. – ranah Sep 02 '20 at 16:44